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EXECUTIVE  SUMMARY 


The  Reduced  Vertical  Separation  Minima  (RVSM)  experiment  resulted  from  conclusions  drawn 
during  the  26th  Meeting  of  the  North  Atlantic  System  Planning  Group  to  conduct  studies  aimed 
at  achieving  early  implementation  of  RVSM  in  the  North  Atlantic  (NAT)  Region.  RVSM  is  an 
approved  International  Civil  Aviation  Organization  concept  to  reduce  aircraft  vertical  separation 
from  the  Conventional  Vertical  Separation  Minima  (CVSM)  of  2000  feet  to  1000  feet,  between 
flight  levels  290  to  410,  within  a  designated  portion  of  the  NAT  Region.  In  the  United  States, 
RVSM  simulation  studies  are  being  conducted  by  the  Federal  Aviation  Administration  (FAA) 
Technical  Center's  National  Simulation  Capability  (NSC)  RVSM  Experimentation  Working 
Group. 

Experimental  procedures,  findings,  and  conclusions  from  Phase  I  of  the  NSC  RVSM  simulations 
are  provided  to  assist  the  FAA’s  Air  Traffic  organizations  in  defining  geographical  areas  for 
RVSM  transitioning  and  establishing  procedures  to  effect  that  transition.  Phase  I  investigated 
workload  changes  resulting  from  the  transition  of  westbound  aircraft  from  RVSM  to  CVSM 
before  leaving  defined  nonradar  RVSM  airspace.  A  second  phase  is  planned  to  investigate 
controller  workload  effects  and  the  feasibility  of  radar  sector  transitions  with  mixed  traffic  flows. 

Real-time  simulations  were  conducted  in  December  1993  and  January  1994  at  the  FAA 
Technical  Center's  Oceanic  Development  Facility.  The  simulation  investigated  nine  different 
scenarios  exploring  four  traffic  conditions.  The  New  York  Air  Route  Traffic  Control  Center 
(ZNY)  Oceanic  Sectors  D71  and  D72  were  configured  wdth  an  Oceanic  Display  and  Planning 
System  position  to  replicate  controller  operations,  including  simulated  high  frequency  (HF)  and 
inter-  and  intra-facility  communications.  Flight  strip  posting  and  Aeronautical  Radio 
Incorporated  (ARINC)  printer  capabilities  were  also  replicated  to  produce  a  high  fidelity, 
functional  control  environment.  Westbound  oceanic  traffic  scenarios  were  developed  based  on 
actual  traffic  flows  recorded  by  the  ZNY.  Simulated  traffic  was  controlled  by  currently  certified 
and  active  ZNY  oceanic  controllers.  Simulated  HF  communication  lines  were  staffed  by  ARINC 
operators. 

Extensive  audio  and  video  recordings  were  made  to  provide  objective  data  of  all  simulation  runs. 
Guided  post-simulation  discussions  and  questionnaires  were  used  to  gather  subjective  data.  A 
dynamic  workload  probe  was  used  at  15-minute  intervals  to  assess  levels  of  workload  throughout 
the  4  to  6  hours  of  each  run. 

The  simulation  results  indicated  that,  while  interval  increases  in  controller  workload  occurred 
under  RVSM  traffic  conditions  when  compared  to  CVSM  conditions,  the  overall  controller 
workload  did  not  increase.  High  interval  vy^orkload  did  not  interfere  with  a  controller’s  ability  to 
provide  service  to  the  aircraft.  Interval  increases  can  be  attributed  to  the  following  causes: 

a.  Additional  coordination  activities  with  adjacent  sectors  and  facilities;  and 

b.  Increased  traffic  scanning  due  to  more  altitudes  being  available  and  occupied. 
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When  controllers  experienced  extremely  high  workload  conditions  during  the  simulation  trials, 
the  nature  of  their  difficulties  with  RVSM  under  complex,  high-volume  traffic  loads  can  be 
summarized  as  follows: 

a.  Keeping  abreast  of  aircraft  position  reports  (updates)  and  other  flight  strip  management 
activities; 

b.  Identifying  the  location  of  an  aircraft  reporting  turbulence;  and 

c.  Extreme  time  pressure  for  traffic  scanning  and  coordination  resulting  in  overload-induced 
operational  errors. 

The  simulation  results  showed  that  the  following  major  traffic  flow  situations  should  be  avoided 
under  RVSM: 

a.  Vertically-stacked  aircraft  with  all  available  altitudes  occupied;  and 

b.  A  traffic  mixture  with  a  high  percentage  of  aircraft  not  flying  track  routes. 

Based  upon  the  Phase  I  RVSM  simulation  results,  the  introduction  of  RVSM  in  the  ZNY 
Oceanic  Airspace  is  feasible  provided  that  certain  procedures  are  well  defined  and  agreed  upon 
prior  to  implementation. 
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1  TNTROnUCTTON. 


This  report  details  procedures  and  findings  from  Phase  I  Reduced  Vertical  Separation  Minima 
(RVSM)  simulations  conducted  by  the  Federal  Aviation  Administration  (FAA)  Technical 
Center's  National  Simulation  Capability  (NSC)  RVSM  Experimentation  Working  Group.  Due  to 
time  constraints,  Phase  I  was  conducted  in  two  parts.  Part  1  and  Part  2. 

RVSM  is  an  approved  International  Civil  Aviation  Organization  concept  to  reduce  aircraft 
vertical  separation  from  the  Conventional  Vertical  Separation  Minima*  (CVSM)  of  2000  feet  to 
1000  feet,  between  flight  levels  (FL)  290  to  410,  within  a  designated  portion  of  the  North 
Atlantic  (NAT)  Region. 

1.1  BACKGROUND. 

The  RVSM  experiment  stems  from  a  decision  made  during  the  26th  Meeting  of  the  North 
Atlantic  System  Planning  Group  to  conduct  studies  leading  to  early  implementation  of  RVSM  in 
the  NAT  Region. 

The  NSC  RVSM  simulations  were  scheduled  in  phases.  Phase  I  was  designed  to  study  the 
transition  of  westbound  aircraft  from  RVSM  to  CVSM  before  leaving  RVSM/Minimum 
Navigation  Performance  Specification  (MNPS)  airspace.^  Aircraft  that  enter  radar  coverage 
adjacent  to  RVSM/MNPS  airspace  or  Canadian  airspace,  which  is  either  MNPS  or  radar 
coverage,  are  an  exception  (see  figure  1). 

1.2  PURPOSE. 

The  simulation  conditions  were  designed  to  provide  baseline  data  for  a  comparison  between 
current  oceanic  operations  under  CVSM  and  planned  air  traffic  (AT)  operations  under  RVSM. 
These  data  will  be  used  to  provide  the  agency's  AT  service  organizations,  especially  the 
International  Procedures  Branch  (ATP- 140),  with  vital  human  performance  information  needed 
to  define  issues  in  the  RVSM  implementation  process. 

1.3  APPROACH. 

Data  collection  efforts  were  designed  to  measure  changes  in  controller  workload  across  various 
conditions  and  focused  on  procedural  issues  that  arose  from  the  controllers’  simulation 
experiences  with  RVSM  operations. 


1  CVSM  -  2,000  ft.  VSM  above  FL  290  up  to  FL  600,  inclusive. 

2  MNPS  airspace  -  A  portion  of  the  NAT  airspace  between  FL275  and  FL400  extending  between  latitude  27N  and  the  North  Pole, 
bounded  in  the  east  by  eastern  boundaries  of  control  areas  (CTAs)  Santa  Maria  oceanic,  Shanwick  oceanic,  Reykjavik,  and  in  the  west 
by  the  western  boundaries  of  CTA  Gander  oceanic  and  the  western  boundary  of  CTA  New  York  oceanic,  excluding  the  area  west  of 
60W  and  south  of  38.4N.  To  ensure  safe  separation,  aircraft  operating  in  this  airspace  must  meet  a  specified  minimum  navigation 
performance  standard. 
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FIGURE  1.  PHASE  I  TRANSITION  AREA 
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2.  SUBJECTS. 


Simulation  participants  included  four  Full  Performance  Level  (FPL)  controllers  (referred  to 
hereafter  as  controllers  A,  B,  C,  and  D)  from  the  New  York  Air  Route  Traffic  Control  Center 
(ZNY),  and  two  technical  observers  (T/Os),  also  from  the  ZNY,  (referred  to  as  observers  X  and 
Y)  with  air  traffic  control  (ATC)  training  experience.  Participation  in  all  simulation  exercises 
was  voluntary.  Data  records  were  managed  by  assigning  four-digit  numbers  to  each  individual  to 
avoid  disclosure  of  participant  identity. 

The  average  age  of  the  controllers  was  3 1  years  and  their  average  years  of  experience  as  FPL 
controllers  was  3.3.  Almost  all  of  that  experience  was  accumulated  in  a  NAT  oceanic  control 
position.  The  four  controllers  had  their  entire  ATC  experience  at  ZNY.  The  average  age  of  the 
T/Os  was  34.  Both  observers  were  current  as  air  traffic  controllers  (see  appendix  A  for  more 
controller  background  information). 

2.1  BACKGROUND  SURVEY. 

A  background  survey  was  conducted  to  collect  basic  demographic  data  and  some  information 
regarding  the  controllers'  and  observers'  opinions  (and  biases)  regarding  high  workload 
situations.  Data  from  the  background  survey  were  used  as  a  baseline  for  comparison  with  survey 
responses  collected  throughout  the  simulation. 

2.1.1  Background  Survey  Results. 

Each  controller  was  asked  to  indicate  three  aspects  of  ATC  in  the  current  oceanic  environment 
that  they  would  like  to  see  changed  (see  appendix  A).  Their  answers  reflected  the  following 
three  primary  concerns:  having  an  accurate  visual  display  of  aircraft  position,  having  more 
reliable  air-to-ground  and  ground-to-ground  communications,  and  having  additional  personnel 
available  at  the  sectors  during  periods  of  heavy  traffic. 

The  controllers  were  also  asked  about  their  experiences  with  high  workload  conditions.  They 
identified  the  following  factors  as  contributing  to  high  workload:  adverse  weather  conditions, 
flight  strip  printing  speed,  low  flight  progress  strip  board  space,  and  communication  line  outages. 
Although  they  did  not  agree  on  which  of  these  factors  occurred  most  frequently,  all  indicated  that 
slow  strip  generation  causes  a  "domino  effect"  on  workload. 

Controllers  and  observers  were  presented  with  10  factors  that  experimenters  believed  contributed 
to  high  workload  in  the  current  ATC  environment  (see  appendix  A  for  a  complete  listing  of 
questions  and  results).  Participants  were  asked  to  check  any  items  they  felt  contributed 
significantly  to  high  levels  of  workload.  In  addition,  they  were  asked  to  add  any  factors  not 
listed.  The  following  summarizes  the  responses  of  the  controllers  and  observers; 

a.  100%  checked  Flight  Information  Regions  (FIRs); 

b.  83%  marked  Oceanic  Display  and  Planning  System  (ODAPS)  printer  speed; 
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c.  50%  of  the  participants  selected  "Coordination  with  Fellow  Controllers";  and 

d.  17%  commented,  "Sometimes  coordination  can  take  several  minutes  because  of 
inexperienced/weak  control  personnel." 

3.  SIMULATION  DESIGN. 

Phase  I  simulations  were  conducted  in  the  Oceanic  Development  Facility  (ODF)  located  at  the 
FAA  Technical  Center.  Due  to  time  constraints.  Phase  I  was  scheduled  in  two  parts.  Part  1  was 
conducted  December  6-10, 1993,  and  Part  2  followed  on  January  10-14, 1994.  As  a  result  of 
controller  comments  during  Part  1,  Part  2  employed  different  scenario  characteristics  and 
controller  rotation.  For  both  parts,  the  physical  environment  of  the  ODF  realistically  simulated 
the  ZNY,  including  the  available  equipment  and  communication  interfaces  (see  figure  2).  The 
ODF,  together  with  the  ODAPS,  which  is  also  located  at  the  Technical  Center,  provided  a 
complete  simulated  oceanic  ATC  environment. 

3.1  ROUIPMENT/CONFIGURATION. 

The  ODF  was  configured  to  include  two  complete  oceanic  control  positions,  each  containing  an 
M-1  console.  The  console  included:  strip  bays,  integrated  Flight  Data  Input/Output  (FDIO) 
equipment,  voice  communication  equipment,  flight  strip  printers.  Aeronautical  Radio 
Incorporated  (ARINC)  printers  (emulated  by  dot  matrix  printers),  and  overhead  sector  charts. 
During  the  Phase  I  simulations,  only  one  ODAPS  Plan  View  Display  (PVD)  was  used. 

A  Target  Generator  (TG)  resident  in  the  ODF  utilized  flight  plan  and  adaptation  data  to  generate 
aircraft  targets.  The  TG  allowed  aircraft  to  react  dynamically  to  controller-issued  clearances. 
Position  reports  that  represented  the  trajectory  of  simulated  aircraft  were  also  provided  by  the 
TG. 

Two  Digital  Equipment  Corporation  work  stations  were  configured  as  Remote  Operator  (RO) 
positions.  Both  were  used  to  simulate  voice  communications  between  ARINC,  adjacent  sectors, 
FIRs,  and  the  controllers.  One  RO  position  had  the  additional  capabilities  of  modifying  flight 
plan  data  resident  in  the  TG  and  of  generating  and  directing  ARINC  messages  to  the  appropriate 
sector  positions. 

ODAPS  performed  flight  data  processing  for  the  simulated  ZNY  Oceanic  Sectors  D71  and  D72. 
It  processed  flight  plan  data  and  related  messages,  in  conjunction  with  stored  adaptation  data,  to 
produce  outputs  that  were  transmitted  via  an  FDIO  control  unit  to  FDIO  equipment  located  at  the 
oceanic  sector  positions  in  the  ODF.  The  FDIO  equipment  used  the  data  output  by  ODAPS  to 
print  flight  strips  and  other  essential  messages  at  the  appropriate  sector  position.  ODAPS  also 
provided  controllers  with  a  graphical  representation  of  flight  plan-extrapolated  positions  of  all 
aircraft  under  their  control. 
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RVSM  Simulation  Configuration 


A  Combine  Sector  message  was  entered  into  ODAPS  to  combine  sector  DTl's  FDIO,  PVD,  and 
flight  strip  information  with  D72's  at  the  D72  position.  A  system-build  restore  tape  was 
generated  from  ZNY’s  Adaptation  Controlled  Environment  Subsystem  tapes  which  were  current 
at  the  time  of  the  simulation.  The  reference  number  for  the  restore  tape  is  TC0284.  The  software 
loaded  on  the  Series  1  was  National  Airspace  Data  Interchange  Network  for  the  ODAPS 
Communication  System  and  SA2000  for  the  ODAPS  Display  Channel. 

1?.  SCENARIOS. 

All  scenarios  were  developed  in  cooperation  with  a  ZNY  ATC  specialist  referred  to  in  this  report 
as  the  scenario  developer.  Phase  I  scenarios  were  developed  from  Data  Analysis  and  Reduction 
Tool  (DART)  runs  of  System  Analysis  and  Recording  (SAR)  tapes  from  ZNY.  The  DART  was 
run  against  the  SAR  tapes  to  extract  flight  plans,  amendment  messages,  ARINC  messages,  and 
upper  winds  information.  Actual  flight  plans  were  loaded  onto  a  personal  computer  for  manual 
editing  to  create  simulated  flight  plans.  After  simulated  flight  plans  were  generated,  they  were 
entered  into  the  scenario  generation  tool  resident  in  the  ODF.  The  tool  put  the  flight  plan 
information  in  a  format  suitable  for  simulation. 

3.2.1  SAR  T ape  Information. 

Part  1  scenarios  were  developed  based  on  flight  plan  data  extracted  from  a  ZNY  ODAPS  SAR 
tape  dated  March  18, 1993,  between  1400  Greenwich  Mean  Time  (GMT)  and  1800  GMT.  Part  2 
scenarios  used  existing  Part  1  information  and  additional  flight  plan  data  from  a  ZNY  ODAPS 
SAR  tape  dated  December  15, 1993,  between  1000  GMT  and  2000  GMT.  The  SAR  tapes 
provided  flight  information  which  was  used  to  recreate  primarily  westbound  traffic  patterns  for 
ZNY  Oceanic  Sectors  D7 1  and  D72  in  a  simulated  environment. 

3.2.2  Traffic  Type. 

To  allow  for  valid  comparisons,  each  traffic  scenario  consisted  of  97  aircraft  traversing  sectors 
D71  and  D72.3  Days  2,  3, 4,  5, 7,  and  9  were  developed  from  the  4  hours  of  recorded  ZNY 
traffic  during  March.  Due  to  the  traffic  density  of  this  day,  ZNY  recommended  this  traffic  flow 
be  expanded  to  a  6-hour  period  (expanded  traffic  conditions)  to  better  replicate  a  moderate  flow. 

For  days  6,  8,  and  10,  flight  plan  information  was  extracted  from  10  hours  of  December  SAR 
tape  traffic  and  condensed  to  create  a  4.5-hour  scenario  (condensed  traffic  conditions).  There 
were  two  reasons  for  condensing  the  traffic  and  using  a  different  SAR  tape  for  portions  of  the 
Part  2  simulations.  First,  controllers  commented  that  March  scenarios  were  not  an  accurate 
reflection  of  traffic  that  was  current  at  the  time  of  the  simulation  because  the  flow  patterns  were 
no  longer  accurate  and  traffic  was  not  heavy  enough.  Second,  the  scenario  developer  followed 
the  assumption  that  fewer  delays  would  be  placed  on  aircraft  departing  European  airports 


3  Training  scenario  not  included. 
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because,  under  RVSM  rules,  there  would  be  more  altitudes  available  resulting  in  a  more 
condensed  traffic  flow;  therefore,  traffic  would  be  heavier. 

3.2.3  Scenario  Conditions. 

The  first  day  of  simulation  was  a  training  day.  Subsequent  days  consisted  of  simulations  under 
CVSM  and  RVSM  rules,  respectively.  There  were  a  total  of  9  experimental  conditions  that 
varied  by  traffic  scenario  (March  93  or  December  93),  traffic  flow  (expanded  or  condensed), 
separation  rules  in  effect  (CVSM  or  RVSM),  and  by  pilot  reports  of  clear  air  turbulence  (CAT). 
Table  1  shows  the  sequence  of  experimental  conditions  followed  throughout  the  2-week 
simulation. 


TABLE  1.  SEQUENCE  OF  EXPERIMENTAL  CONDITIONS 
FOR  PHASE  I  SIMULATIONS 


Conditiont 

Part 

Run  Date 

Description 

2 

CVSM-1 

1 

12/7/93 

Expanded  Scenario,  CVSM  Rules, 

March  93  traffic 

3 

CVSM-2 

1 

12/8/93 

Expanded  Scenario,  CVSM  Rules, 

March  93  traffic 

B 

RVSM-1 

1 

12/9/93 

Expanded  Scenario,  RVSM  Rules, 

March  93  traffic 

5 

RVSM-2 

1 

12/10/93 

Expanded  Scenario,  RVSM  Rules, 

March  93  traffic 

6 

RVSM-3 

2 

1/10/94 

Condensed  Scenario,  RVSM  Rules, 

December  93  traffic 

B 

MTT-1 

2 

1/11/94 

Minimum  Time  Track,  Expanded  Scenario, 
RVSM  Rules,  March  93  traffic 

8 

MTT-2 

2 

1/12/94 

Minimum  Time  Track,  Condensed  Scenario, 
RVSM  Rules,  December  93  traffic 

9 

CAT-1 

2 

1/13/94 

Clear  Air  Turbulence  Reports,  Expanded 
Scenario,  RVSM  Rules,  March  93  traffic 

10 

CAT-2 

2 

1/14/94 

Clear  Air  Turbulence  Reports,  Condensed 
Scenario,  RVSM  Rules,  December  93  traffic 

See  following  sections  for  detailed  description  of  conditions. 


3.2.3. 1  Training  Scenario- 

On  the  first  day  of  simulation,  a  training  scenario  was  used  to  introduce  participants  to  the  ODF. 
It  consisted  of  a  20-aircraft  scenario  with  a  minimal  amount  of  estimates  and  ARINC 
communications.  CVSM  rules  were  in  effect  and  March  traffic  was  used. 
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:^.2.3.2  CVSM  Scenarios. 


On  simulation  days  2  and  3,  scenarios  were  conducted  using  CVSM-1  and  CVSM-2  conditions 
(respectively)  to  provide  a  baseline  for  comparison  to  RVSM  conditions.  The  expanded  March 
traffic  was  used.  These  2  days  utilized  ZNY  operating  procedures  current  at  the  time  of  the 
simulation. 

3.2.3.3  RVSM  Scenarios. 

On  simulation  days  4  and  5,  RVSM-1  and  RVSM-2  conditions  utilized  expanded  March  traffic 
scenarios  and  applied  operating  procedures  current  at  the  ZNY  at  the  time  of  the  simulation,  with 
the  exception  of  reduced  vertical  separation.  On  Day  6,  RVSM-3  was  conducted  with  the  same 
procedures  as  in  the  previous  RVSM  runs,  but  a  condensed  traf  fic  scenario  was  used. 

3  2.3.4  Minimum  Time  Track  fMTTf  Scenarios. 

Operationally,  the  ZNY  makes  use  of  MTTs  to  accommodate  heavy  traffic  flows.  "Because  of 
the  constraints  of  large  horizontal  separation  criteria  and  a  limited  economical  height  band 
(FL3 10-390)  the  airspace  [NAT  Region]  is  very  congested  at  peak  hours.  In  order  to  provide  the 
best  service  to  the  bulk  of  the  traffic,  a  system  of  organized  tracks  is  constructed  every  12  hours 
to  accommodate  as  many  aircraft  as  possible,  on  or  close  to,  their  minimum  time  paths.”"* 

Since  MTTs  are  an  important  part  of  the  ZNY  operations,  scenarios  were  developed  to  simulate 
MTT  traffic  flows.  The  tracks  used  for  MTT-1  and  MTT-2  conditions  were  the  published 
westbound  tracks  used  in  the  NAT  region  on  March  18,  1993,  and  December  15,  1993, 
respectively.  These  tracks  were  established  to  accommodate  as  many  westbound  aircraft  leaving 
Europe  on,  or  close  to,  their  minimum  time  paths. 

MTT-1  and  MTT-2  conditions,  which  were  used  on  simulation  days  7  and  8,  are  modifications  of 
RVSM-1  and  RVSM-3  conditions,  respectively.  As  table  2  illustrates,  the  MTT  scenarios  had  a 
larger  percentage  of  aircraft  moved  onto  tracks.  All  vertical  separation  rules  that  were  applied  to 
RVSM  were  applied  to  MTT  (see  section  3.2.5  and  appendix  B  for  track  descriptions). 

3 .2.3 .5  CAT  Scenarios. 

The  last  experimental  condition,  CAT,  was  conducted  on  the  final  2  days  of  simulations.  This 
consisted  of  simulation  runs  similar  to  RVSM  until  two  pilot  reports  of  greater  than  moderate 
CAT  were  received  by  controllers.  When  this  situation  oecurred,  the  controller  was  required  to 
comply  with  the  ATC  in-flight  contingency  procedures  for  RVSM  airspace  defined  in  the  NAT 
Vertical  Separation  Implementation  Group  Meetings  One^  and  Two.^ 


4  Reference  "North  Atlantic  MNPS  Airspace  Operations  Manual  Sixth  Edition",  January  1994. 

5  November  4-8,  1991,  FAA,  Washington,  DC. 

6  March  2-6,  1992,  FAA,  Washington,  DC. 
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TABLE  2.  AIRCRAFT  ON  TRACKS 


Track  Identification 

Condition^ 

G 

H 

J 

K 

L 

M 

N 

No  Track 

%  on  Tracks 

RVSM-1 

24 

5 

8 

5 

iiHS 

46 

53% 

MTT-1 

31 

0 

13 

5 

B 

BBI 

42 

57% 

RVSM-3 

N/A 

N/A 

N/A 

B 

0 

1 

83 

14% 

MTT-2 

N/A 

N/A 

N/A 

N/A 

m 

14 

14 

56 

42% 

f  Refer  to  section  3.2.3. 3  for  description  of  RVSM  conditions. 


During  the  CAT-1  pre-simulation  briefing,  controllers  were  informed  of  the  in-flight  contingency 
procedures  that  applied  to  RVSM  airspace.  If  more  than  one  aircraft  reported  greater  than 
moderate  CAT,  the  controllers  were  instructed  to  take  the  following  actions: 

a.  If  the  reporting  aircraft  has  reduced  vertical  separation  with,  and  is  within  5  minutes  of, 
another  aircraft,  then  the  controller  should  establish  a  conventional  separation  by 
climbing/descending  either  aircraft. 

b.  Any  aircraft  not  yet  cleared  on  the  affected  track/level  should  be  conventionally  separated 
for  the  remainder  of  the  active  period  of  the  Oceanic  Track  System  time  parameter. 

The  purpose  of  these  simulation  runs  was  to  examine  the  controller's  ability  to  safely  and 
effectively  handle  a  CAT  situation. 

The  CAT-1  condition  used  on  simulation  day  9  was  primarily  the  RVSM-1  scenario  with  pilot 
reports  of  greater  than  moderate  CAT.  The  final  day  (CAT-2  condition)  was  a  slight 
modification  of  the  condensed  MTT-2  scenario  with  CAT  reports. 

3.2.4  Sector  Boundaries. 

To  emulate  an  actual  operational  environment  and  to  allow  for  an  equitable  distribution  of 
workload,  the  boundary  between  sectors  D71  and  D72  changed  on  a  daily  basis  (see  table  3  and 
appendix  B).  Controllers  were  informed  each  day  of  the  sector  boundary  in  effect. 

3.2.5  Tracks. 

In  the  NAT  region,  an  organized  track  structure  exists  to  provide  the  best  service  to  the  bulk  of 
the  traffic.  When  high  traffic  levels  are  expected,  EUR/CAR’  tracks  are  established  in  addition 


7  Track  established  to  cater  to  the  European/Carribean  (EUR/CAR)  axis. 
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TABLE  3,  ASSIGNED  SECTOR  BOUNDARIES  PER  CONDITION 


Day 

Condition* 

D71/72  Sector  Boundarj 
Latitude/Longitude  in  degrees 

J 

2 

CVSM-1 

38/40 

37/50 

37/60 

BDAtf 

3 

CVSM-2 

38/40 

37/50 

37/60 

BDA 

4 

RVSM-1 

38/40 

37/50 

37/60 

BDA 

5 

RVSM-2 

38/40 

37/50 

37/60 

BDA 

6 

RVSM-3 

37/40 

37/50 

37/60 

BDA 

7 

MTT-1 

38/40 

37/50 

33/60 

BDA 

8 

MTT-2 

40/40 

38/50 

37/60 

BDA 

9 

CAT-1 

40/40 

38/50 

37/60 

BDA 

10 

CAT-2 

27/40 

37/50 

37/60 

BDA 

I  Refer  to  section  3.2.3  for  a  description  of  conditions. 
Bermuda  Control  Area 


to  NAT  tracks.  These  routes  differ  from  "  'core'  tracks  in  that  they  may  cross,  and  in  some  cases 
may  not  extend  from  coast  out  to  coast  in."* 

The  westbound  NAT  and  EUR/CAR  tracks  published  on  March  1 8, 1 993  and  December  1 5, 

1993  were  included  in  each  scenario.  These  tracks  are  shown  in  table  4  and  appendix  B. 

3.2.6  ARINC  Messages. 

Different  but  equal  numbers  of  ARINC  messages  were  scripted  for  each  simulation  day  except 
Day  5,  RVSM-2.  The  scenario  developer  expected  the  increase  in  available  altitudes  under 
RVSM  to  increase  pilot  requests  for  higher  altitudes.  RVSM-2  was  used  to  test  controllers  under 
this  situation  and  thus  was  scripted  with  a  higher  number  of  ARINC  messages. 

The  send  time  and  sector  designation  for  each  message  were  determined  from  flight  plan  data 
and  sector  boundary  information.  ARINC  messages  consisted  mostly  of  pilot  requests  for  higher 
altitudes.  Messages  were  delivered  via  the  RO  position  to  the  proper  simulated  ARINC  printer. 
A  monitor  was  assigned  to  track  aircraft  altitudes  and  positions  to  ensure  scripted  ARINC 
messages  were  appropriate  throughout  each  scenario. 

Because  of  controller  comments  during  initial  debriefings,  the  scenario  developer  decided  to 
incorporate  additional  “requests  for  higher”  for  aircraft  flying  at  or  below  FL310.  The  additional 
requests  for  low  altitude  aircraft  and  the  monitors'  omissions  of  some  messages  caused  a  daily 
fluctuation  in  the  number  of  ARINC  messages  (see  table  5). 


8  Reference  "North  Atlantic  MNPS  Airspace  Operations  Manual  Sixth  Edition",  January  1994. 
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TABLE  4.  TRACK  POSITIONS  BY  SIMULATION  DAY 


Idev 

Condition 

Track 

HRH 

Track  Coordinates  I 

1 

Training 

G 

NAT 

49/15  49/2  47/30  46/40  44/50  42/60  POGGO 

H 

NAT 

48/08  48/15  48/20  46/30  45/40  43/50  41/60  JOBOC 

K 

46/28  46/15  45/20  43/30  38/40  34/45  30/50  25/55  20/60 

8/60 

J 

EUR7CAR 

47/08  47/15  47/20  45/30  43/40  40/50  35/60  PRISS 

H 

NAT 

DETNA  42/20  43/30  44/40  42/50  40/60  SLATN 

CVSM-1 

SAME  AS  ABOVE 

rr 

CVSM  -2 

SAME  AS  ABOVE 

B 

RVSM-1 

SAME  AS  ABOVE 

RVSM-2 

SAME  AS  ABOVE 

6 

RVSM-3 

■■ 

NAT 

4130/15  42/20  43/30  44/40  44/50  43/60  POGGO 

M 

NAT 

40/15  40/20  41/30  42/40  43/50  42/60  JOBOC 

N 

IRiSM 

41/40  39/50  34/60  BDA 

1 

MTT-1 

G 

NAT 

49/15  49/20  47/30  46/40  44/50  42/60  POGGO 

H 

NAT 

48/08  48/15  48/20  46/30  45/40  43/50  41/60  JOBOC 

N 

EUR/CAR 

41/40  39/50  34/60  BDA 

MTT-2 

Hi 

NAT 

4130/15  42/20  43/30  44/40  44/50  43/60  POGGO 

M 

NAT 

43/15  42/20  42/30  41/40  39/50  34/60  BDA 

N 

EUR/CAR 

FS  35/40  27/50  18/57 

9 

CAT-1 

G 

NAT 

49/15  49/20  47/30  44/40  44/50  42/60  POGGO 

H 

NAT 

48/08  48/15  48/20  48/30  43/40  43/50  41/60  JOBOC 

J 

47/08  47/15  47/20  45/30  39/40  37/50  35/60  PRISS 

K 

EUR/CAR 

46/28  46/15  45/20  43/30  38/40  34/45  30/50  25/55 

L 

NAT 

DETNA  42/20  43/30  42/40  42/50  40/60  SLATN 

10 

CAT-2 

L 

NAT 

4130/15  42/20  43/30  44/40  44/50  43/60  POGGO 

TABLE  5.  NUMBER  OF  SCRIPTED  ARINC  MESSAGES  SENT  PER  CONDITION 


1  Condition 

Total  Messages 
Sent 

%  Messages  Sent 
toD71 

%  Messages  Sent 
toD72 

CVSM-1 

34 

56 

44 

CVSM-2 

54 

37 

63 

RVSM-1 

54 

44 

56 

RVSM-2 

78 

37 

63 

RVSM-3 

52 

60 

40 

MTT-1 

45 

31 

69 

MTT-2 

52 

56 

44 

CAT-1 

53 

43 

57 

CAT-2 

57 

58 

42 

^  ?.7  Planned  Events. 

Planned  events  were  also  scripted  for  each  day.  These  consisted  of  any  estimates,  altitude 
revisions,  or  time  revisions  issued  by  adjacent  sectors  or  FIRs.  Planned  events  were  sent  by  the 
RO  position  via  phone  commumcations.  While  the  total  number  of  aircraft  in  each  scenario 
remained  constant,  the  number  of  handoff  estimates  given  per  phone  call  was  varied  to  emulate 
realistic  intrafacility  communications.  This  resulted  in  a  different  number  of  phone  calls  per 
scenario.  Table  6  presents  the  frequency  of  phone  calls  made  to  both  sectors. 

SIMULATION  SUPPORT  PERSONNEL- 

Flight  strips  were  posted  by  support  personnel  on  movable  strip  bays.  Strips  were  posted  at 
controller  sectors  by  either  a  T/0  or  other  ZNY  personnel.  The  RO  positions  were  staffed  by  two 
ARINC  radio  operators  and  by  NSC  support  personnel. 

^  4  FTDFJ.TTY  ISSUES. 

There  were  several  differences  between  the  simulated  and  actual  oceanic  environment,  including 
the  physical  properties  of  the  OD  APS  position  and  phone  lines,  communication  response  times, 
and  variation  in  traffic  patterns.  A  detailed  list  of  differences  and  their  magnitude  are  shown  in 
table  7. 


Efforts  were  made  to  correct  any  reported  scenario  and  communication  differences.  Because  of 
time  constraints,  the  hardware  and  lab  configuration  remained  unchanged. 
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TABLE  6.  FREQUENCY  OF  PHONE  CALLS  MADE  TO  SECTORS  D71  AND  D72t 


Condition 

Total  Calls 

%  Calls  to  D71 

%  Calls  to  D72 

CVSM-1 

24 

29 

71 

CVSM-2 

33 

21 

79 

RVSM-1 

35 

29 

71 

RVSM-2 

43 

28 

72 

RVSM-3 

29 

52 

48 

MTT-1 

28 

21 

79 

MTT-2 

29 

41 

59 

CAT-1 

24 

33 

67 

CAT-2 

27 

33 

67  1 

t  Does  not  include  inter-sector  communication. 

TABLE  7.  SIMULATION  FIDELITY:  BASED  ON 
CONTROLLER/OBSERVER  COMMENTS _ 


Magnitude 
of  Reported 
Difference 

Category 

Description 

Comparison  between  ZNY  and  the  ODF  simulated 
environment 

SLIGHT 

Equipment 

Different  Phones 

SLIGHT 

Equipment 

Faster  ARINC  printer  speed  in  the  ODF  lab 

SLIGHT 

Scenario/Event 

Longer  delays  in  RO  phone  answering  at  ZNY 

MODERATE 

Scenario/Event 

More  estimates  received  at  one  time  at  ZNY 

SLIGHT 

Traffic  Realism 

More  Aircraft  reports  of  "unable"  at  ZNY 

MODERATE 

Traffic  Realism 

More  requests  by  Aircraft  for  "higher"  at  ZNY 

MODERATE 

Traffic  Realism 

Aircraft  accept  fewer  climbs  at  ZNY 

MODERATE 

Position  of  ODAPS  to  the  right  of  "D"  at  ZNY 

MODERATE 

Strip  bay  configuration  (space)  insufficient  in  the  ODF  lab 

3.4.1  Physical  and  Functional  Realism. 

Controllers  and  observers  were  asked  to  rate  the  physical  (equipment)  and  functional  (operations) 
realism  of  the  ODF  using  their  current  ZNY  environment  for  comparison.  Realism  was  rated  on 
a  scale  from  I  (Very  Low)  to  10  (Very  High).  Table  8  shows  the  means  and  standard  deviations 
of  the  ratings  given  by  controllers  and  observers  for  each  simulation  day  and  the  simulation  as  a 
whole. 
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TABLE  8.  PHYSICAL  AND  FUNCTIONAL  REALISM  OF  THE  SIMULATION 


Physical  Realism 

Functional  Realism  j 

Condition 

Mean 

Std.  Dev. 

Mean 

Std.  Dev. 

Training 

7.00 

1.00 

6.40 

1.14 

CVSM-1 

7.50 

1.38 

7.30 

1.03 

CVSM-2 

7.67 

1.37 

7.17 

0.98 

RVSM-1 

7.83 

0.75 

8.20 

0.84 

RVSM-2 

6.67 

1.51 

6.67 

1.51 

RVSM-3 

7.80 

0.84 

8.20 

1.10 

MTT-1 

7.30 

1.11 

7.67 

1.51 

MTT-2 

7.50 

1.38 

7.83 

1.60 

CAT-1 

7.17 

1.57 

7.50 

1.98 

CAT-2 

7.50 

1.22 

8.00 

1.26 

TOTAL 

7.40 

1.23 

7.49 

1.36 

Controllers  gave  a  high  rating  for  the  2  weeks  of  simulation.  Physical  and  functional  realism 
both  had  an  overall  mean  of  at  least  7.4.  In  the  post  simulation  discussions,  controllers 
commented  that  the  following  list  of  items  were  different  from  ZNY,  but  they  did  not  think  the 
items  made  any  significant  difference  in  the  results  of  the  simulation  experiments  : 

a.  The  number  of  available  strip  bays  were  fewer  [the  ODF  uses  roll-aways]. 

b.  The  ODAPS  position  would  be  located  between  D71  and  D72  at  22^ Y. 

c.  The  ARINC  printers  in  the  ODF  were  quieter. 

4.  EXPERIMENTAI.  PROCEDURES. 

The  following  sections  detail  the  simulation  procedures,  data  types,  and  collection  methods  used 
for  Phase  1 . 

4.1  TRAINING. 

During  the  training  day,  controllers  were  familiarized  with  the  various  functions  in  the  ODF, 
questionnaires,  and  procedures  for  dynamic  workload  probes.  Controllers  also  completed  a  brief 
background  information  questionnaire  which  queried  their  experiences  with  high  traffic  loads 
and  high  workload. 

4.2  CONTROT-T-ER  ASSIGNMENT. 

Controller  position  assignments  varied  between  Parts  1  and  2.  All  controllers  were  rotated 
through  all  three  control  positions  (D71,  D72,  and  ODAPS)  to  minimize  scenario  familiarity  and 
to  maximize  the  number  of  samples  per  position.  For  Part  1 ,  the  same  individual  was  denoted  as 
controller  A,  B,  C,  or  D  or  as  observer  X  or  Y.  Controller  C  could  not  participate  in  the  Part  2 
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simulation  runs  due  to  schedule  conflicts.  Part  1  Observer  Y  was  assigned  to  work  Controller 
C’s  vacant  position.  No  additional  observers  or  controllers  participated  in  Part  2. 

4.2.1  Part  1  Assignment. 

During  the  four  experimental  days  of  Part  1,  CVSM-1,  and  2,  and  RVSM-1  and  2,  each 
controller  was  assigned  to  a  rotating  schedule  consisting  of  a  continuous  90-minute  work  session 
followed  by  a  30-minute  break.  The  controllers  followed  the  same  rotation  scheme  through  the  3 
control  positions  (D71,  D72,  and  ODAPS)  for  each  of  the  4  days.  The  rotation  assignment  is 
shown  in  table  9. 

TABLE  9.  PART  1  CONTROLLER  POSITION  ASSIGNMENTS 
BY  HALF-HOUR  PERIODS 


Period 

Sector  D71 
(Observer  X) 

Sector  D72 
(Observer  Y) 

ODAPS 

(XandY) 

On  Break  | 

1 

A 

C 

B 

D 

2 

D 

C 

B 

A 

3 

D 

C 

A 

B 

4 

D 

B 

A 

C 

5 

C 

B 

A 

D 

6 

C 

B 

D 

A 

7 

C 

A 

D 

B 

8 

B 

A 

D 

C 

9 

B 

A 

C 

D 

10 

B 

D 

C 

A 

11 

A 

D 

C 

B 

Since  controller  assignment  remained  constant  over  the  four  experimental  days  of  Part  1,  the 
same  individual  worked  the  same  control  position  during  the  same  interval  for  each  day.  For 
example,  controller  C  always  worked  position  D72  during  the  third  interval.  The  schedule 
allowed  a  comparison  of  the  various  conditions  based  on  an  individual's  data  because  the  amount 
of  traffic  and  communication  load  was  roughly  comparable.  The  rotation  assignment  was  used 
to  maximize  the  power  of  the  analysis  because  there  were  only  four  subjects. 

4.2.2  Part  2  Assignment- 

To  maximize  controller  exposure  to  control  positions,  balance  task  load,  and  minimize  controller 
scenario  familiarity,  controllers  were  randomly  assigned  to  a  given  position  for  each  day  of  Part 
2.  The  rotation  assignment  for  Part  2  is  shovm  in  table  10. 
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TABLE  10.  PART  2  CONTROLLER  POSITION  ASSIGNMENTS 
BY  HALF-HOUR  PERIODS 


RVSM-3 

MTT-1 

MTT-2 

CA 

T-1 

CA1 

^-2 

Period 

D71 

D72 

ODAPS 

D71 

D72 

ODAPS 

D71 

D72 

ODAPS 

D71 

D72 

ODAPS 

D71 

D72 

ODAPS 

n 

A 

B 

C 

B 

B 

B 

C 

A 

B 

B 

A 

C 

B 

B 

A 

A 

B 

C 

B 

B 

B 

C 

A 

B 

B 

A 

C 

B 

H 

A 

3 

X 

H 

C 

A 

B 

X 

C 

X 

B 

B 

A 

X 

B 

B 

A 

B 

X 

B 

C 

B 

B 

X 

A 

X 

B 

B 

C 

X 

C 

B 

A 

X 

B 

B 

B 

B 

X 

A 

X 

C 

A 

C 

X 

C 

B 

B 

c 

B 

B 

B 

B 

c 

H 

B 

C 

A 

C 

B 

C 

H 

B 

B 

c 

B 

B 

B 

B 

c 

A 

B 

C 

A 

C 

B 

c 

H 

B 

c 

B 

B 

X 

B 

c 

X 

B 

C 

A 

X 

B 

X 

H 

B 

9 

c 

B 

A 

B 

B 

c 

X 

B 

A 

C 

X 

B 

X 

A 

C 

10 

B 

B 

A 

B 

B 

A 

X 

C 

A 

C 

X 

A 

.  X 

B 

C 

11 

B 

B 

A 

B 

B 

A 

B 

C 

A 

c 

B 

A 

A 

B 

c  1 

During  Part  2  simulations,  controllers  generally  worked  continuously  for  2  hours  at  an  assigned 
position.  All  other  procedures  were  identical  to  those  executed  in  Part  1 . 

4A  RATING  FORMS. 

The  following  section  describes  the  controller  and  observer  pre-  and  post-simulation 
questionnaires  and  data  recording  forms  used  during  each  simulation  run.  The  questionnaires, 
data  recording  forms,  and  instructions  are  included  in  appendix  C. 

4  A .  1  Dynamic  Quick  Observer  Assessments. 

The  following  measures  were  taken  at  1 5-minute  intervals  throughout  the  simulation  runs: 

a.  Workload  ratings  given  verbally  by  the  controllers  to  the  observers.  The  rating  scale 
ranged  from  1  (very  low)  to  10  (very  high)  perceived  workload; 

b.  Workload  ratings  by  the  observers  using  the  same  10-point  scale; 

c.  Observer  ratings  ofcontroller  coordination  and  communication.  A  rating  of  1  indicated 
remarkably  good  and  5  indicated  unusually  poor; 
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d.  Observer  ratings  of  traffic  management  and  control  judgment,  using  the  same  5 -point 
scale;  and 

e.  Observer  ratings  of  controller  flight  strip  management  using  the  5-point  scale.’ 

All  the  ratings  were  obtained  from  the  two  sector  control  positions.  In  addition,  workload  ratings 
were  obtained  from  the  ODAPS  position.  Observer  X  sat  or  stood  behind  the  D71  control 
position  and  observed  the  controller,  while  tallying  various  activities.  At  15-minute  intervals,  the 
observer  probed  the  controller  for  his  or  her  workload  rating.  The  observer  then  made  a  rating 
judgment  for  the  same  15-minute  time  interval.  Observer  Y  performed  these  tasks  for  both  the 
D72  sector  and  the  ODAPS  positions.  When  an  observer  was  not  available  (i.e.,  on  break), 
experimental  support  staff  assumed  the  responsibility  of  collecting  controller  ratings;  however, 
they  did  not  make  any  other  judgment  assessments. 

4.3.2  Post-Simulation  Questionnaires. 

Following  each  simulation  day,  the  controllers  and  observers  met  to  complete  post-simulation 
questionnaires  and  to  discuss  issues  surrounding  that  day's  simulation.  The  discussions  were 
audio  taped.  Post-simulation  questionnaires  consisted  of  three  types  of  questions.  The  first  type 
addressed  the  fidelity  and  realism  of  the  simulation  environment.  The  second  question  group 
dealt  with  overall  ratings  of  workload  and  operational  performance.  The  third  was  concerned 
with  anticipated  or  experienced  differences  between  CVSM  and  RVSM  operations.  The 
questionnaires  served  to  focus  the  topics  of  the  discussions. 

4  4  AT  IDTO  AND  VIDEO  DATA  RECORDING. 

An  extensive  audio  and  video  system  was  used  for  data  collection  during  each  simulation  run. 
Four  separate  camera  views  were  recorded,  consisting  of  sectors  D71  and  D72,  the  ODAPS 
control  area,  and  an  overall  view  of  the  ODF.  The  video  was  recorded  in  Super  VHS  format  on 
2-hour  tapes  and  was  stamped  with  National  Television  System  Committee  time  code  for 
synchronous  playback. 

Eleven  separate  audio  channels  were  recorded  on  a  1/2-inch  audio  tape.  The  audio  signals 
recorded  were  obtained  from  three  wireless  microphones  worn  by  the  controllers,  two  ambient 
microphones,  and  six  channels  of  controller  headsets  and  speakers  used  in  the  ODF  voice 
communication  system.  Several  audio  channels  were  also  mixed  and  recorded  on  the  Super  VHS 
recorders'  audio  tracks  to  match  the  appropriate  camera  view. 

Three  separate  camera  views,  along  with  the  associated  audio,  were  sent  from  the  ODF  to  the 
Human  Factors  Laboratory  (HFL)  on  fiber  optic  lines  and  displayed  on  three  large  screens.  The 
HFL  briefing  room  is  a  remote  location  at  the  FAA  Technical  Center.  This  display  allowed 
observers  to  view  the  experiment  in  real  time  without  intrusion,  thereby  maintaining  a  sterile  and 
non-obtrusive  simulation  environment  in  the  ODF. 

’  The  observer  forms  were  changed  slightly  from  Part  I  to  Part  2  simulations  to  accommodate  recorder  preferences. 
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5.  RESULTS. 


The  reported  findings  were  based  on  controller  concerns  that  arose  during  debriefings  and  from 
questionnaires.  Statistical  data  analysis  were  performed  to  support  or  disclaim  subjective 
comments  when  appropriate.  All  statistical  tests  were  performed  with  a  significance  level  a  = 
0.05. 

5.1  OPERATIONAL  ERRORS. 

Table  1 1  lists  the  operational  errors  observed  and  recorded  by  the  technical  observers  (T/Os) 
during  the  simulation  runs.  No  errors  were  recorded  during  both  Conventional  Vertical 
Separation  Minima  (CVSM)  runs,  both  Minimum  Time  Track  (MTT)  runs,  and  the  Clear  Air 
Turbulence  (CAT)-2  run.  Several  errors  were  recorded  during  the  Reduced  Vertical  Separation 
Minima  (RVSM)  runs  and  the  CAT-1  run. 

TABLE  1 1 .  OPERATIONAL  ERRORS  NOTED  BY  THE  T/OS 


Day 

Interval 

Explanation 

CVSM-1 

None  noted. 

CVSM-2 

- 

None  noted. 

RVSM-1 

9 

Plane  progressed  at  FL300.  Strip  was  marked  FL330. 

10 

Strip  not  marked  at  FL290. 

13 

Aircraft  reported  FL300  at  38/40.  Strip  marked  as  FL320. 
Position  report  written  on  45W  strip  not  40W  strip. 

RVSM-2 

11 

ODAPS  missed  an  incorrect  altitude  on  the  40W  position 
report  for  AOM433.  Approved  at  FL300,  activated  at 
original  strip  altitude,  FL350. 

8 

30/60  aircraft  coordinated  at  FL370,  actual  altitude  was 
FL360  -  strip  flat  in  bay  -  indicating  complete  (MPH633). 

RVSM-3 

7 

CFG600  given  clearance  "maintain  FL360  until  61 W 
climb  to  cross  62W  at  FL370."  This  did  not  allow  aircraft 
to  be  level  in  MNPS  airspace. 

14 

AAL69  coordinated  at  31/60  time  17:19  FL350.  Aircraft 
was  only  7  minutes  from  boundary.  AAL69  strip  laid  flat 
and  was  not  coordinated  with  BDA. 

15 

IBE6101  at  30/60.  Did  not  coordinate  with  D89  until  1 
minute  prior  to  progressing  60W. 

MTT-1 

- 

None  noted. 

MTT-2 

- 

None  noted. 

CAT-1 

IBH 

Loss  of  Separation,  9  minutes  at  60W. 

17 

Aircraft  climbing  to  FL370.  When  that  aircraft  reported 
out  of  FL360  another  aircraft  was  cleared  to  climb.  This 
violated  Air  Traffic  Procedures  (ATP)  report  leaving 
procedures. 

CAT-2 

- 

None  Noted. 

Note:  Controllers  reported  that  the  traffic  was  so  dense  that 

they  often  were  uncertain  about  lateral  separation  between 
aircraft. 
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S  RFFRCTS  OF  RVSM  ON  SERVICE  TO  AIRCRAFT. 


Controllers  claimed  there  was  a  decrease  in  aircraft  service  under  RVSM  conditions.  They 
reported  a  greater  ability  to  approve  altitude  requests  under  CVSM  conditions  compared  to 
RVSM.  Under  RVSM  conditions,  controllers  felt  they  were  more  likely  to  issue  pilot  advisories 
of  unable  to  climb/descend,  or  to  simply  ignore  pilot  requests.  This  was  due  to  traffic  and  their 
own  heavy  workload. 

During  each  simulation  run,  the  frequency  and  transmission  times  of  pilot  requests  for  higher 
altitudes  were  recorded.  Aircraft  position  reports,  providing  time  and  altitude  information  for 
each  aircraft,  were  extracted  from  S AR  tapes  created  during  the  simulation.  Position  reports, 
pilot  requests,  and  the  number  of  aircraft  descents  were  the  available  data  utilized  to  assess  the 
service  supplied  to  the  airlines. 

Aircraft  that  climbed  directly  to  a  requested  altitude  soon  after  a  request  was  made  were  assumed 
to  have  moved  in  response  to  that  request  (referred  to  as  “direct  move”).  If  no  movement  was 
reported  by  a  requesting  aircraft,  it  was  assumed  that  the  request  was  ignored  or  denied  (referred 
to  as  “no  move”).  Since  there  were  no  scripted  pilot  requests  for  descents,  aircraft  descents  were 
assumed  to  be  issued  by  controllers  as  a  result  of  traffic  conditions.  Finally,  if  a  requesting 
aircraft  climbed  to  a  level  other  than  what  was  requested,  it  was  assumed  to  be  either  a  result  of 
traffic  or  the  only  clearance  a  controller  could  issue  in  response  to  that  request.  This  type  of 
climb  was  not  analyzed  because  the  reason  for  the  movement  was  indeterminate. 

The  following  analyses  exclude  both  CAT  runs.  Although  RVSM  was  employed  during  the 
CAT  testing,  service  to  the  aircraft  was  not  measured  because  contingency  procedures  were  in 
effect.  Aircraft  could  have  been  descended  or  had  clearances  denied  due  to  weather  conditions. 

The  5  days  of  RVSM  runs  were  combined  and  compared  to  the  2  CVSM  baseline  runs.  The 
purpose  was  to  determine  if  RVSM  was  a  significant  factor  in  granting  or  denying  an  aircraft's 
request  for  a  higher  altitude,  and  to  see  if  a  controller  was  more  likely  to  issue  an  unrequested 
descend  clearance  under  RVSM. 

Request  and  movement  frequencies  for  CVSM  and  RVSM  are  presented  in  table  12.  A  test  of 
proportions,  using  arc  sine  transformation,  was  performed  on  CVSM  and  RVSM  for  no 
movement  and  direct  movement,  n'  =  121 .3,  =  .212.  For  No  Moves,  \  =  .062  <  /i,  showed  no 

statistically  significant  difference  between  separation  minima.  The  same  held  for  Direct  Moves, 
/j.  =  .106  <  /ij  This  indicated  that  a  controller  was  not  more  likely  to  deny  or  grant  a  request  for 
higher  altitudes  under  CV SM  conditions  than  RV SM  conditions. 

Because  no  significant  difference  was  found  for  separation  minima,  traffic  type  was  explored  in 
an  attempt  to  explain  controller  comments.  Table  13  reorganized  the  data  in  table  12  by  traffic 
type. 
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TABLE  12.  PROPORTION  OF  REQUESTS  RESULTING  IN  DIRECT 
AND  NO  MOVEMENT  BY  SEPARATION  MINIMA 


Direct  Moves 

No  Moves  1 

CVSM 

.35 

_ . 

RVSM 

.30 

.47  _J 

TABLE  13.  PROPORTION  OF  REQUESTS  RESULTING  IN  DIRECT 
AND  NO  MOVEMENT  BY  TRAFFIC  TYPE 


Traffic  Type  ^ 

Direct  Moves 

No  Moves  1 

Expanded 

.34 

.44 

Condensed 

.23 

.55  1 

^  See  section  3.2.2  for  more  information  on  traffic  types. 


A  test  of  proportions,  using  arc  sine  transformation,  was  performed  on  expanded  and  condensed 
traffic  for  no  movement  and  direct  movement,  n'  =  145.5,  =  .194.  For  No  Moves,  =  .220  > 

\  showed  a  statistically  significant  difference  between  traffic  types.  A  significant  difference  was 
also  found  for  Direct  Moves,  =  .220  >  h^.  Unlike  the  test  of  proportions  done  on  data  grouped 
by  separation  minima,  a  test  by  traffic  type  showed  statistical  significance.  This  indicated  that  a 
controller  was  more  likely  to  deny  or  grant  a  request  for  higher  altitudes  under  condensed  traffic 
conditions  than  expanded  traffic  conditions,  regardless  of  separation. 

Examination  of  table  14  further  supported  the  above  findings.  The  highest  number  of  descend 
clearances,  7,  was  issued  under  condensed  traffic.  All  other  runs  had  between  0  and  2  descend 
clearances. 


TABLE  14.  DESCEND  CLEARANCES  ISSUED  PER  RUN 


Traffic  Type 

Average 

Descend 

Range 

Expanded 

1.2 

0.2 

Condensed 

7 

7 

^  INCREASED  USE  OF  WHEN  ABLE  HIGHER  fWAHf  INFORMATION. 

Controllers  expressed  a  need  for  information  about  an  aircraft's  ability  to  climb  to  a  higher 
altitude  in  order  to  develop  an  effective  strategy  for  handling  traffic,  especially  under  high 
workload  conditions.  This  was  claimed  to  be  particularly  true  when  issuing  clearances  to  aircraft 
operating  at  relatively  higher  RVSM  altitudes  in  comparison  to  a  CVSM  altitude.  Controllers 


20 


preferred  to  obtain  this  information  before  a  high  workload  situation  occurred.  This  would  allow 
time  for  other  necessary  tasks  rather  than  obtaining  WAH  information  with  time  consuming 
telephone  conversations. 

The  bar  chart  in  figure  3  illustrates  the  average  number  of  all  requests  made  by  controllers 
compared  to  the  average  number  that  were  for  WAH  only.  Averages  depicted  in  the  figure  were 
tabulated  from  manually  recorded  phone  conversations  between  the  controllers  and  Remote 
Operator  (RO)  positions.  Inspection  of  figure  3  revealed  a  notable  increase  in  the  average 
percentage  of  WAH  requests  under  all  scenarios  run  with  RVSM  compared  to  CVSM.  Twelve 
percent  of  all  requests  made  during  CVSM-2  were  for  WAH,  as  compared  to  an  average  of  41 
percent  under  RVSM  conditions.  Even  higher  averages  were  observed  under  MTT  and  CAT 
conditions. 

5.4  WORKLOAD  ASSESSMENT. 

Throughout  the  simulation,  interval  workload  was  assessed  by  the  controllers  and  the  T/Os.  A 
non-parametric  statistical  correlation  (Spearman's  Rank)  was  calculated  to  evaluate  the 
relationship  between  the  controllers’  and  observers’  ratings.  A  very  high,  statistically  significant 
positive  correlation  was  observed,  =  .75,  p  <  .001. 

Figure  4  shows  the  95  percent  confidence  interval  (Cl)  of  the  average  controller  interval 
workload  ratings  as  recorded  on  the  Quick  Forms  each  day  (see  section  4.3.1  and  appendix  C).  It 
can  be  seen  from  figure  4  that  the  highest  average  interval  workload  ratings  were  observed  vinder 
CVSM.  All  scenario  parameters  remained  constant  (i.e.,  traffic  type,  percent  of  traffic  on  tracks) 
for  the  RVSM-1/-2  and  CVSM-1/-2  runs.  Each  day  the  average  interval  workload  ratings 
decreased  under  RVSM,  showing  that  the  introduction  of  RVSM  did  not  increase  controller 
workload.  For  RVSM-3,  scenario  parameters  were  modified  and  the  average  interval  ratings 
increased,  but  still  did  not  exceed  the  CVSM  ratings.  Also,  it  should  be  noted  that  for  all 
conditions,  the  day  employing  condensed  traffic  had  a  slightly  higher  average  interval  workload 
rating  than  for  the  day  with  expanded  traffic,  (i.e.,  MTT-2,  p  =3.8  >  MTT-1  p  =  3.5). 

A  non  parametric  test  (Mann- Whitney  Test)  on  the  average  interval  workload  ratings  showed  a 
statistically  significant  decrease  rmder  RVSM  (p  =  3.77)  compared  to  CVSM  (p  =  4.25), 

Z  =  -2.47,  p  =  .014.  However,  the  highest  interval  workload  rating  (10)  was  reported  under 
RVSM.  Controllers  explained  that  the  increase  in  available  altitudes  xmder  RVSM  created  longer 
flight  strip  scanning  times  and  unfamiliarity  of  altitude  patterns,  which  resulted  in  high  interval 
workload  ratings. 

An  overall  rating  at  the  end  of  each  run  was  recorded.  Controller  and  observer  overall  average 
ratings  ranged  from  6.5  to  6.9  for  all  conditions. 
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WORKLOAD  RATING 


CONDITION 

Note:  Due  to  technical  complications,  an  accurate  account  of  requests  made  during  CVSM-1 
was  not  calculated,  thus  CVSM-1  results  were  not  reported. 

FIGURES.  ALL  REQUESTS  AND  WAH  REQUESTS 


CVSM-1  RVSM-1  RVSM-3  MTT-2  CAT-2 


CVSM-2  RVSM-2  MTT-1  CAT-1 


I  95%  Cl 

■  Mean  Controller 


DAY 


FIGURE  4.  95  PERCENT  Cl  OF  CONTROLLER  ASSESSED  WORKLOAD  RATINGS 
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5.4.1  Workload  Based  on  ARINC  Printer  Paper- 


Experienced  Air  Traffic  Control  (ATC)  personnel  claimed  that  the  length  of  paper  rolled  out 
from  an  Aeronautical  Radio  Incorporated  (ARINC)  printer  on  the  control  room  floor  is  an 
informal  metric  used  to  instantly  gauge  the  relative  level  of  controller  workload.  The  total  length 
of  an  ARINC  printer  page  is  determined  by  the  number  of  messages  sent  to  the  printer  before  a 
controller  tears  the  paper.  The  total  number  of  messages  that  appeared  on  each  sheet  of  ARINC 
paper  tom  during  the  simulation  was  recorded.  Since  each  ARINC  message  had  to  be  read  and 
possibly  required  flight  strip  updates  or  ATC  action,  paper  scraps  gave  observers  access  to  a 
readily  available  and  intuitive  measure  of  the  amount  of  task  loading  on  the  controllers. 

The  length  (i.e.,  number  of  messages)  of  a  piece  of  tom  ARINC  paper  was  assumed  to  be 
partially  dependent  on  how  busy  a  controller  was  with  other  control  activities  and  the  relative 
level  of  workload  experienced  at  the  time  messages  were  sent.  The  correlation  coefficient  for 
reported  controller  workload  and  the  maximum  number  of  messages  on  a  piece  of  paper,  rs  = 
.3508,  p  <  .01,  revealed  a  very  high  positive  correlation.  This  indicated  that  higher  workload 
ratings  were  associated  with  a  larger  number  of  messages  per  piece  of  paper.  Accordingly, 
ARINC  printer  paper  length  constituted  an  objective  measure  of  workload. 

When  expanded  traffic  was  run  with  CVSM,  the  average  number  of  messages  was  3.7.  When  the 
same  traffic  was  run  with  RVSM,  the  average  rose  slightly  to  4.0.  However,  when  traffic  was 
condensed  and  run  with  RVSM,  the  average  rose  to  5.3.  A  Mest  performed  on  messages  by 
traffic  type  resulted  in  statistical  significance,  t  =  -2.32,  p  =  .02 1 .  The  same  traffic  run  with 
RVSM  or  CVSM  had  similar  average  workloads.  Tests  with  complex  traffic  resulted  in  a  higher 
controller  workload. 

5.4.2  Effects  ofTraffic  on  Workload. 

During  pre-simulation  discussions,  controllers  speculated  that  having  a  greater  percentage  of 
traffic  flying  track  routes  would  reduce  their  workload,  presumably  because  they  could 
incorporate  the  advanced  track  knowledge  into  their  control  plan.  The  controllers  stated  that  they 
preferred  a  traffic  flow  that  consisted  of  approximately  60  to  75  percent  of  aircraft  on  track 
routes.  During  debriefing  sessions,  they  revealed  that  the  advanced  marking  of  flight  strips  with 
the  track-identifying  letter  and  having  the  tracks  shown  as  a  background  feature  on  the  Oceanic 
Display  and  Planning  System  (ODAPS)  display  mitigated  high  workload  conditions.  The 
percentage  of  aircraft  flying  track  routes  and  the  average  workload  ratings  tabulated  from  the 
Quick  Form  (see  section  4.3.1)  are  displayed  in  table  15. 

The  correlation  between  the  amount  of  aircraft  flying  track  routes  and  the  average  D71  workload 
ratings,  rs  =  -.41,  p  <  .001,  showed  a  highly  significant  negative  correlation.  This  strong  inverse 
relationship  revealed  that  the  D71  controller  reported  being  less  busy  when  a  higher  percentage 
of  the  total  aircraft  were  flying  track  routes  and  more  busy  when  a  high  percentage  were  flying 
random  routes.  This  trend  was  apparent  when  conditions  were  examined  separately  within  the 
sector.  For  example,  D7rs  workload  increased  from  an  average  of  3.33  for  RVSM-2  to  6.53  for 
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TABLE  15.  AVERAGE  WORKLOAD  RATINGS  AND  PERCENTAGE  OF  AIRCRAFT 

FLYING  TRACK  ROUTES  PER  DAY 


Day 

Aircraft  on 
Tracks 

Sector  D71 
Workload 
Rating 

Sector  D72 
Workload 
Rating 

CVSM-1 

53  % 

5.08 

4.58 

CVSM-2 

53% 

4.36 

5.72 

RVSM-1 

53% 

4.32 

5.09 

RVSM-2 

53% 

3.33 

4.35 

RVSM-3 

14% 

6.53 

3.05 

MTT-1 

57% 

3.11 

5.21 

MTT-2 

42% 

5.67 

3.72 

CAT-1 

54% 

3.65 

4.80 

CAT-2 

12% 

5.31 

4.15 

RVSM-3.  RVSM-3  and  CAT-2  had  the  lowest  percentages  of  aircraft  flying  track  routes,  14 
percent  and  12  percent  respectively,  and  the  highest  average  workload  values  reported  for  D71 
were  during  these  days. 

In  contrast,  the  correlation  for  the  average  workload  ratings  for  the  D72  controllers  and  the 
number  of  aircraft  flying  track  routes  was  significantly  positive,  n  =  .35,  p  <  .001.  This 
relationship  revealed  that  the  D72  controller  reported  being  busier  when  a  high  percentage  of  the 
total  aircraft  were  flying  track  routes.  Again,  this  trend  was  apparent  when  conditions  were 
examined.  It  should  be  noted,  however,  that  the  magnitude  of  the  decreased  average  workload 
within  conditions  for  D72  was  smaller  than  the  magnitude  of  increased  average  workload  for 
D71. 

These  opposite  trends  can  be  explained  by  the  track  structures  used  for  Phase  I.  For  all  days,  a 
larger  number  of  tracks  were  routed  through  sector  D72  than  D71 .  Therefore,  an  increase  in 
track  traffic  increased  the  traffic  through  sector  D72  and  decreased  the  traffic  flow  through  D71 , 
resulting  in  the  observed  changes  in  average  workload  ratings  for  each  sector. 

An  assessment  of  track  percentage  versus  average  workload  without  regard  to  sector  was 
performed.  CVSM-1  and  CVSM-2  were  excluded  from  this  analysis  since  the  track  percentage 
was  identical  for  both.  Also,  RVSM-1  was  excluded  because  it  had  the  same  track  percentage  as 
RVSM-2.  Figure  5  illustrates  that  the  overall  effect  of  increased  track  traffic  was  decreased 
workload. 

In  each  condition,  it  appeared  that  workload  was  highly  related  to  the  relative  distribution  of 
aircraft  flying  track  routes  and  random  routes.  When  fewer  aircraft  were  flying  on  tracks,  the 
track  pre-planning  knowledge  was  less  effective,  and  the  overall  workload  increased. 
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FIGURE  5.  AVERAGE  WORKLOAD  BY  CONDITION 


5.5  COMMUNICATIONS. 

Controllers  reported  that  RVSM  runs  required  more  coordination  between  sectors  D72  and  D71 
to  work  out  climb  clearances.  Also  under  RVSM,  especially  MTT  and  CAT,  controllers  reported 
having  a  difficult  time  coordinating  with  each  other  because  one  or  another  was  on  the  phone. 
The  amoimt  of  time  spent  on  the  phone  receiving  handoff  estimates  and  a  random  sample  of  time 
controllers  spent  on  the  phone  in  general  were  extracted  from  audio/video  recordings  of  the 
simulation. 

Figure  6  illustrates  the  95  percent  Cl  for  the  average  time  it  took  for  a  controller  to  receive  one 
handoff  estimate  given  by  an  adjacent  sector  or  FIR. 

/ 

A  non-parametric  test  (Kruskal-Wallis)  showed  a  significant  difference  in  average  times  across 
conditions,  18.13,  p  =  .000.  The  smallest  average  time,  31.67  seconds,  was  observed  imder 
CVSM,  and  the  numbers  increased  to  as  high  as  43.62  under  MTT.  Controllers  stated  that  the 
increased  number  of  altitudes  under  RVSM  resulted  in  longer  flight  strip  scanning  times.  This 
possibly  accoimted  for  the  increase  in  time  to  receive  a  handoff  estimate. 

A  non-parametric  test  (Mann- Whitney)  of  the  same  data  grouped  by  traffic  type  showed  a 
significant  difference  in  average  times  between  traffic  types,  Z  =  -3.28,  p  =  .001 .  An  average 
time  of  34.78  seconds  was  observed  when  traffic  was  expanded.  The  average  rose  to  40.88 
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FIGURE  6.  MEAN  TIME  FOR  A  HANDOFF  ESTIMATE 
TO  BE  RECEIVED  BY  CONTROLLERS 

seconds  when  traffic  was  condensed,  indicating  increased  time  to  receive  a  handoff  estimate 
when  traffic  was  condensed.  This  trend  was  observed  within  each  condition.  For  example, 
MTT-2  (condensed)  had  a  higher  average  time  than  MTT-1  (expanded). 

Figure  7  illustrates  the  95  percent  Cl  for  the  average  elapsed  time  for  a  random  sample  of 
controller  phone  conversations. 

A  Kruskal- Wallis  Test  revealed  a  significant  difference  in  the  length  of  a  phone  conversation 
between  conditions,  17.56,  p  =  .001.  Examination  of  figure  7  showed  virtually  no  difference 
between  CVSM  (p  =  37.3)  and  RVSM  (p  =  37.2).  Average  times  greatly  increased  under  the 
MTT  (p  =  49.3)  and  CAT  (p  =  5 1 . 1)  conditions,  supporting  controller  claims. 

The  average  number  of  calls  controllers  made  per  hour  to  adjacent  sectors  and  FIRs  was 
calculated.  The  smallest  average  was  observed  under  MTT,  43  calls  per  hour.  RVSM  had  the 
largest  average,  56  calls  per  hour.  Controllers  commented  that  less  coordination  was  necessary 
when  more  aircraft  were  flying  track  routes,  (i.e.,  MTT).  Also,  aircraft  exiting  sector  D72  to  the 
north  did  not  need  to  be  transitioned  back  to  a  CVSM  altitude.  A  majority  of  the  published 
tracks  during  the  MTT  runs  were  in  the  northern  part  of  D72.  The  RVSM  runs  had  more  random 
routing. 


During  debriefing  sessions,  controllers  also  mentioned  that  when  operating  under  RVSM,  more 
information  needed  to  be  relayed  to  the  relieving  controller  which  resulted  in  longer  position 
relief  briefings.  This  information  included  conveying  information  about  which  aircraft  needed  to 
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FIGURE  7.  MEAN  TIME  OF  A  CONTROLLER  PHONE  COMMUNICATION 


be  moved  out  of  their  altitudes.  Lengths  of  the  position  relief  briefings  were  not  recorded, 
therefore,  data  is  reported. 


6  DISCUSSION. 

Controllers  were  instructed  to  transition  aircraft  from  RVSM  to  CVSM  before  leaving  the 
RVSM/  Minimum  Navigation  Performance  Specification  (MNPS)  airspace,  with  the  exception 
of  aircraft  that  entered  radar  coverage  adjacent  to  RVSM/MNPS  airspace  or  Canadian  airspace. 
The  following  findings  and  recommendations  are  based  on  controllers’  simulation  experience 
with  RVSM  transitioning  in  nonradar  airspace. 

Controllers'  unfamiliarity  with  new  altitudes  and  traffic  patterns  attributed  to  the  operational 
errors  observed  during  the  RVSM  runs.  After  limited  exposure  to  RVSM  through  simulation, 
controllers  felt  RVSM  could  be  effectively  implemented  with  minimal  errors.  However,  in  order 
for  controllers  to  safely  and  effectively  operate  under  RVSM,  clearly  defined  procedures  are 
required.  The  following  issues  should  be  addressed; 

a.  When  reverting  back  to  CVSM  from  RVSM,  is  legal  separation  defined  by  2000  feet 
between  aircraft  regardless  of  altitude  (odd  or  even)  or  only  by  2000  feet  using  conventional 
altitudes? 

b.  Since  there  is  no  instantaneous  reversion  to  CVSM,  what  is  legal  and  how  should  this  be 
accomplished  (i.e.,  can  controllers  climb  several  aircraft  at  once)? 

c.  During  a  CAT  situation,  do  all  aircraft  in  the  airspace  need  to  be  reverted  back  to 
conventional  altitudes,  or  should  only  the  reporting  aircraft  be  removed  from  the  turbulent 
altitude? 


27 


In  addition  to  ensuring  safe  control  of  traffic,  the  controllers'  abilities  to  effectively  service 
aircraft  were  examined.  The  introduction  of  RVSM  did  not  reduce  the  quality  of  service  to  the 
airlines.  When  either  separation  (RVSM  or  CVSM)  was  applied  to  the  same  traffic  situation, 
controllers  were  just  as  likely  to  deny  or  approve  pilot  requests.  In  fact,  the  availability  of  more 
altitudes  allowed  more  aircraft  to  fly  at  or  near  their  optimum  altitudes.  It  was  mentioned, 
however,  that  better  altitude  assignment  would  be  possible  if  all  aircraft  operating  in  the  MNPS 
airspace  were  RVSM-certified  and  equipped.  This  would  allow  controllers  to  apply  minimum 
separation  to  all  aircraft,  thus  providing  more  availability  for  optimum  altitudes. 

When  trying  to  devise  an  effective  traffic  plan  under  RVSM,  controllers  found  a  greater  need  for 
WAH  information  than  they  did  under  CVSM.  Due  to  the  increase  in  communications  necessary 
to  receive  such  information,  WAH  information  should  be  available  without  a  controller  rec;uest. 
Some  concerns  about  the  possible  impact  of  having  to  request  WAH  were  raised  during  de¬ 
briefing  sessions.  They  included  a  greater  occurrence  of  confusion  among  foreign  pilots  between 
requests  for  altitude  information  and  actual  clearances,  and  the  unavailability  and  inaccuracy  of 
remarks  about  WAH  in  the  International  Civil  Aviation  Organization  flight  plans.  Controllers 
suggested  ways  to  make  WAH  available  to  controllers  without  request.  These  included:  creating 
a  coordinator  position  at  all  ATC  centers  to  alleviate  controllers  of  some  of  their  less  critical 
duties j  having  Santa  Mana  Air  Route  Traffic  Control  Center  (ARTCC)  include  WAH 
information  when  they  pass  estimates  to  New  York  ARTCC  (ZNY);  and  having  pilots  report 
WAH  information. 

Controller  workload  was  a  primary  concern  of  this  simulation.  Analyses  indicated  that  although 
interval  increases  in  workload  were  observed,  the  overall  impact  of  RVSM  resulted  in  no 
increase  in  controller  workload  compared  to  CVSM.  Controllers  mitigated  high  interval 
workload  by  allowing  the  ODAPS  controller  to  be  more  active  in  traffic  coordination  and  climbs. 

When  faced  with  the  potential  for  high  workload  conditions,  controllers  stated  that  having 
sufficient  time  for  pre-planning  and  having  advanced,  detailed  knowledge  of  the  track  structure 
for  that  day  was  important  for  managing  the  traffic  load.  The  amount  of  pre-planning  a 
controller  could  accomplish,'  based  on  track  knowledge,  effected  his/her  ability  to  manage  high 
traffic  loads.  Discerning  from  this,  aircraft  flying  random  non-track  routes  seemed  to  demand 
more  of  the  controllers’  resources  than  aircraft  flying  track  routes.  As  a  result,  controllers 
suggested  minimizing  random  routing  to  help  alleviate  high  interval  workload  levels. 

Diuing  simulation,  controllers  experienced  additional  differences  between  RVSM  and  CVSM. 
Under  RVSM,  controllers  experienced  longer  position  relief  briefings,  longer  flight  strip 
scanning  times  due  to  unfamiliarity  of  altitude  patterns,  and  in  some  cases,  increased 
communications.  As  a  result,  controllers  made  some  suggestions:  assign  the  ODAPS  controller 
more  responsibility  to  help  coordinate  traffic  and  to  assist  with  climbs,  create  a  planner  position 
at  the  Centers  to  start  an  off-loader  track  and  to  relieve  controllers  of  less  critical  duties  during 
busy  periods;  allow  transitions  to  occur  in  radar  sectors;  split  sectors  on  a  daily  basis  considering 
track  positions;  and  minimize  random  routing. 
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The  increased  available  altitudes  under  RVSM  provided  greater  flexibility  for  managing  traffic. 
These  altitudes  are  most  effective  if  all  aircraft  operating  in  RVSM  airspace  are  certified  and 
equipped.  Based  upon  the  Phase  I  RVSM  simulation  results,  the  introduction  of  RVSM  in  New 
York  ARTCC’s  Oceanic  Airspace  is  feasible  provided  that  certain  procedures  are  well  defined 
and  agreed  upon  prior  to  implementation. 
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APPENDIX  A 

CONTROLLER  AND  OBSERVER  BACKGROUND  INFORMATION 


Observers: 

Observer  #  5720  =  Observer  Y  [Observer  X  for  Part  II] 
Observer  #  2793  =  Observer  X  [Observer  Y  for  Part  II] 
Controllers: 

Controller  #3009  =  Controller  A  [for  both  Part  I  and  Part  II] 
Controller  #5116  =  Controller  B  [for  both  Part  I  and  Part  II] 
Controller  #2145  =  Controller  C  [Not  present  for  Part  II] 
Controller  #823 1  =  Controller  D  [Controller  C  for  Part  II] 


TABLE  A-l.  CONTROLLER  BACKGROUND  INFORMATION  SUMMARY' 


Controllers: 

Observers: 

Controller:  | 

A 

B 

C 

D 

X 

Y 

Mean 

Mode 

Ages 

25 

29 

29 

41 

34 

34 

31 

29 

Years  FPL 

0 

3 

3 

7 

6 

1 

3.25 

3 

Years  Oceanic 

0 

3 

3 

7 

1 

3.25 

3 

Years  as  NAT  FPL 
Oceanic  Controller 

0 

3 

3 

7 

■ 

1 

3.25 

3 

Months  since  controlling 

I 

oceanic 

0 

0 

0 

0 

0 

0 

0 

0 

Years  as  a  trainer  in 

ATC 

0 

■ 

“ 

10 

5 

3.5 

4.67 

Facilities  worked 

ZNY 

ZNY 

ZNY 

ZNY 

ZNY 

ZNY 

Q 1 .  If  you  had  the  opportunity  to  change  three  current  elements  in  the  oceanic  area  (practices, 
procedures,  equipment,  etc.)  what  would  they  be? 


1  Controller  and  Observer  Designations  refer  to  Part  I  Assignment  Scheme. 
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Al. 


X 

Equipment  -  Some  type  of  radar  or  global  positioning  system. 

Communications  -  direct  contact  with  aircraft. 

Telephone  Systems  -  more  reliable  links  with  other  facilities. 

Y 

Increase  strip  printing  speed. 

Increase  ODAPS  update  rate. 

Consider  a  coordinator  position  to  call  QX  &  PAZ  on  heavy  eastbound  flows. 

A 

Eliminate  grease  pencils  and  Plexiglas  maps,  and  provide  fully  automated  map 
displays. 

B 

Would  like  to  be  on-line  with  foreign  facilities.  This  would  help  to  reduce  the 
time  it  takes  for  coordination. 

C 

Reliable  visual  automation  for  separation  use. 

Reliable  communications  to  aircraft. 

Two  controllers  per  sector  during  heavy  periods  for  safety. 

D 

Equipment.  Accurate  visual  presentation  by  two-way  satellite  link  with  aircraft 
navigation  equipment. 

Procedures.  A  separate  coordination  position  would  increase  sector  efficiency 
during  high  density  traffic. 

Q2.  Based  on  your  experience  with  high  traffic  and  high  workload  in  oceanic  operations  - 
a.  What  are  some  of  the  things  that  can  occur  that  could  cause  a  controller  to  have 
significant  difficulties  in  maintaining  an  orderly  and  expeditious  traffic  flow  ? 
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X 

Weather  is  an  important  factor,  as  are  equipment  problems  (phone  line  outages), 
and  inexperience  of  control  personnel. 

Y 

Strip  generation  backlogged. 

Increasing  pressure  on  approving  altitudes  on  eastbound  flights. 

Falling  behind  on  ARINC  progresses  during  saturation  traffic,  especially  when 
time  revisions  are  routinely  occurring. 

A 

Board  space  is  limited,  overflowing  bays  of  flight  strips  allows  for  mismarking 
altitudes,  times,  etc. 

Random  crossing  traffic  presents  problems. 

Following  successive  strips  on  random  routes  where  flight  paths  of  aircraft  cross 
and  re-cross  sometimes  presents  problems. 

B 

Not  having  flight  strips  on  aircraft  entering  my  sector. 

Coordination  difficulties  (not  answering  the  line). 

C 

Poor  staffing  during  heavy  traffic  periods,  slow  flight  strip  generation. 

D 

Inordinately  long  time  spent  on  the  phone  to  accomplish  manual  coordination. 
Lack  of  board  space. 

Strip  printing. 
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Q2b.  Which  of  these  events  tend  to  occur  most  frequently  ? 

A2b. 

X  Weather.  _ _ 

Y  No  response. _ _ 

A  No  response. _ _ 

B  Not  having  strips. _ 

C  Both. _ _ 

D  Lack  of  board  space. _ 

Q2c.  Which  of  these  events  would  most  likely  cause  additional  problems? 


A2c. 


X 

Telephone  problems. 

Inexperienced  controllers. 

Y 

Backloeeed  strip  generation-  allows  less  time  to  plan  traffic  flow. 

A 

As  air  traffic  increases  and  more  aircraft  are  thrown  into  the  picture,  it  becomes 
increasingly  difficult  to  follow  complex  random  flight  paths  and  to  provide  all 
aircraft  "good"  fuel  efficient  altitudes  without  any  real  automation  to  assist  ATC. 

B 

Not  having  strips  causes  a  domino  effect.  Aircraft  are  spinning  and  the  flow 
(regulation)  of  traffic  deteriorates. 

C 

Both. 

D 

Coordination  phone  time. 

A-3 


Which  items  below  do  you  feel  contribute  significantly  to  high  levels  of  workload  in  the 
current  oceanic  ATC  system  ? 


D  lx  Y  Response 
Frequency 


Printer  Speed  ODAPS 


Printer  Speed  ARINC 


ODAPS  Update  Rate 


ARINC  Com  Delays 


Phone  System 


Special  Pilot  Requests 


Coordination  with 


Foreign  Facilities 


Coordination  with 


Fellow  Controllers 


Oceanic  Track  System 


Random  Routes 


Aircraft  Performance 


Characteristics/Mix 


OTHER 


Additional  Comments 


nr 

Much  time  is  spent  and  lost  because  of  the  items  in  Q1 .  Sometimes  coordination  can 
take  several  minutes  because  of  inexperienced/weak  control  personnel. 

A 

Pilots  often  request  higher  altitudes  and  in  many  cases,  ATC  must  step  climb  other 
aircraft  to  comply  with  pilot  requests.  To  climb  aircraft  and  get  all  necessary  read 
backs  from  ARINC  takes  up  a  great  deal  of  time.  Coordination  with  foreign  facilities, 
for  example,  taking  estimates  (numerous)  from  Santa  Mana,  can  take  as  much  as  10-15 
minutes.  Problems  with  random  routes  are  mentioned  previously. 

B 

The  speed  of  the  ODAPS  printer  affects  everything.  All  traffic  hinges  on  the  strips  . 
which  controllers  can't  get  in  time.  This  also  affects  coordination  with  other  controllers 
and  facilities. 

C 

Strip  generation  lags  during  heavy  traffic  periods.  Coordination  becomes  very  time 
consuming  for  both  sides.  Incorrect  information  from  pilots  (bad  estimate,  wrong 
altitude  report,  etc.)  has  a  snowballing  effect  at  the  sector.  All  of  these  factors  increase 
workload  sharply. 

D 

Foreign  coordinations  take  the  longest  because  other  facilities  use  separate  coordinators 
who  don't  have  the  feeling  of  urgency  a  controller  has. 
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APPENDIX  B 

MAPS,  AIRSPACE,  AND  TRACK  STRUCTURE 


FIGURE  B-2.  PHASE  1  -  PART  2,  RVSM 


B-2 


FIGURE  B-3.  PHASE  1  -  PART  2,  MTT-1 
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FIGURE  B-4.  PHASE  1  -  PART  2,  MTT-2 
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FIGURE  B-5.  PHASE  1  -  PART  2,  CAT-1 
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FIGURE  B-6.  PHASE  1  -  PART  2,  CAT-2 
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APPENDIX  C. 

FORMS  and  QUESTIONNAIRES 

1 .  This  form  is  to  be  completed  by  all  controllers  prior  to  participation  in  the  NSC  Phase  I 
Simulation  activities.  The  form  consists  of  requests  for  general  background  information  and 
an  initial  (baseline)  judgment  regarding  oceanic  control  practices. 

2.  Controllers  should  be  advised  that  their  names  will  not  be  listed  or  appear  in  any  of  the  NSC's 
data  records  to  insure  anonymity  and  to  encourage  unbiased  reporting.  Findings  will  be 
reported  as  group  data  and  genetically,  as  Controller  A,  B,  C,  etc.  In  order  to  facilitate  data 
analysis,  the  experimenters  will  use  the  last  four  digits  of  a  controller’s  social  security  number 
to  collate  various  data  records  belonging  to  a  particular  simulation  subject. 
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NSC  SIMULATION: 


CONTROLLRR  FORM  I. 


GENERAL  BACKGROUND  INFORMATION 

Date: 

T.a<!t  fnnr  digits  of  vour  Social  Security  Number: 

Age: 

How  many  years  of  experience  as  a  fFPL)  controller?  years  months 

How  manv  vears  of  experience  as  an  Oceanic  FPL?  years  months 

How  time  as  an  oceanic  controller  in  the  North  Atlantic  Region? 

Name  the  last  three  facilities  at  which  you  have  worked  starting  with  your  current  assignment: 

tn  (2)  .  (3) 

GENERAL  PROCEDURES  &  PRACTICES  INFORMATION 

1.  If  you  had  an  opportunity  to  change  three  current  elements  in  the  oceanic  area  (practices,  procedures, 
equipment,  etc.),  what  would  they  be? 

2.  Based  on  your  experience  with  high  traffic  and  high  workload  in  oceanic  operations 

a)  What  are  some  of  the  things  that  can  occur  that  could  cause  you  to  have  significant  difficulties  in 

maintaining  an  orderly  and  expeditious  traffic  flow  ? 

b)  Which  of  these  events  tend  to  occur  most  frequently? 

c)  Which  of  these  events  tend  to  most  likely  cause  additional  problems? 

3.  Please  check  all  items  below  that  you  feel  contribute  significantly  to  high  levels  of  workload  in  the  current 
oceanic  ATC  system: 

□  Printer  Speed  ODAPS_  ARINC_  □  Coordination  with  Foreign  Facilities 

□  OD APS  Update  Rate  Dl  Coordination  with  Fellow  Controllers 

□  ARINC  Communication  Delays  □  Oceanic  Track  System 

□  Phone  System  □  Random  Routes 

□  Special  Pilot  Requests  D  Aircraft  Performance  Characteristics/Mix 

□  Other  °  Other 

Please  include  comments  (explanations)  on  any  of  the  above  checked  items: 

NSC  SIMULATION;  OBSERVER  FORM  I. 
INSTRUCTIONS: 


1 .  This  form  is  to  be  completed  by  all  technical  observers  prior  to  participation  in  the  NSC 
Phase  I  Simulation  activities.  The  form  consists  of  requests  for  general  background 
information  and  an  initial  (baseline)  judgment  regarding  oceanic  control  practices. 

2.  Observers  should  be  advised  that  their  names  will  not  be  listed  or  appear  in  any  of  the 
NSC's  data  records  to  insure  anonymity  and  to  encourage  unbiased  reporting.  Findings 
will  be  reported  as  group  data  and  generically,  as  Observer  X,  and  Y.  In  order  to  facilitate 
data  analysis,  the  experimenters  will  use  the  last  four  digits  of  a  controller's  social  security 
number  to  collate  various  data  records  belonging  to  a  particular  simulation  subject. 
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NSC  SIMULATION;  OBSERVER  FORM  I. 


GENERAL  BACKGROUND  INFORMATION 

Date: 

l  ast  four  digits  of  vour  Social  Security  Number: 

Age; 

Hnw  many  years  of  experience  as  a  ('FPL)  controller?  years  months 

Hnw  many  years  of  experience  as  an  Oceanic  FPL?  years  months 

How  much  time  as  an  oceanic  controller  in  the  North  Atlantic  Region? 

How  many  years/months  since  you  have  controlled  oceanic  traffic? 

How  many  years/months  as  a  trainer/deyeloper  in  an  ATC  facility? 

GENERAL  PROCEDURES  &  PRACTICES  INFORMATION 

1 .  If  you  had  an  opportunity  to  change  three  current  elements  in  the  oceanic  area  (practices,  procedures, 
equipment,  etc.),  what  would  they  be? 

2.  Based  on  your  experience  with  high  traffic  and  high  workload  in  oceanic  operations; 

a)  What  are  some  of  the  things  that  can  occur  that  could  cause  you  to  have  significant  difficulties  in 

maintaining  an  orderly  and. expeditious  traffic  flow  ? 

b)  Which  of  these  events  tend  to  occur  most  frequently? 

c)  Which  of  these  events  tend  to  most  likely  cause  additional  problems? 

4 

3.  Please  check  all  items  below  that  you  feel  contribute  significantly  to  high  levels  of  workload  in  the  current 
oceanic  ATC  system: 

□  Printer  Speed  ODAPS_  ARINC_  □  Coordination  with  Foreign  Facilities 

□  ODAPS  Update  Rate  n  Coordination  with  Fellow  Controllers 

□  ARINC  Communication  Delays  □  Oceanic  Track  System 

□  Phone  System  □  Random  Routes 

□  Special  Pilot  Requests  □  Aircraft  Performance  Characteristics/Mix 

□  Other  □  Other 

Comments  on  any  of  the  above  checked  items: 
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NSC  SIMULATION: 


INSTRUCTIONS: 


1 .  This  form  is  to  be  completed  by  all  controllers  after  each  completed  simulation  run  in  the 
NSC  Phase  I  Simulation  activities.  The  form  consists  of  requests  for  information  regarding 
overall  experiences  and  judgments  about  the  just  completed  simulation  run. 

2.  The  RATING  NUMBERS  to  be  used  for  item  #4  are: 

1  =  Remarkably  good 

2  =  Moderately  good 

3  =  So-so 

4  =  Not  very  good 

5  =  Unusually  poor 


3.  Page  2  of  the  Form  consists  of  two  different  versions:  Version  1  is  to  be  filled  out  after 
CVSM  (baseline)  simulation  runs,  Version  2  is  to  be  filled  out  after  simulations  with  RVSM 
conditions.  The  difference  between  these  two  versions  is  that  Version  2  elicits  additional, 
RVSM-specific  information. 
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NSC  SIMULATION:  CONTROLLER  FORM  II 
POST  SIMULATION  RUN  QUESTIONNAIRE 


Simulation  Condition:  _ _ _ 

Your  SS#-(4  last  digits) _ Date: _ Simulation  Run  No: 


1 .  Please  estimate  your  overall  WORKLOAD  during  the  last  simulation  (circle  one)? 


12  34  567  8  9  10 

VERY  LOW  MODERATE  VERY  HIGH 


2.  In  terms  of  REALISM,  how  real  was  the  PHYSICAL  SIMULATION  ENVIRONMENT  for  you 
(circle  one)?  _ _ _ 


12  34  567  8  9  10 

VERY  LOW  MODERATE  VERY  HIGH 


3.  In  terms  of  FUNCTIONAL  REALISM,  how  real  was  the  SIMULATED  TRAFFIC  for  you  (circle 
one)?  _ _ _ _ 


12  3  4  5  67  8  9  10 

VERY  LOW  MODERATE  VERY  HIGH 


4.  Please  judge  your  OWN  PERFORMANCE  over  the  most  recent  simulation  run.  Using  the  RATING 
NUMBERS  from  the  instruction  sheet,  for  each  position  worked,  insert  the  appropriate  rating  into  each 

box  next  to  the  following  factors: _ 

H-POSITION  RADAR-POSITION 


□  Proper  Coordination 

□  Promptness  of  Actions 

□  Situation  Awareness  Maintenance 

□  Communication  Management 

□  Proper  Message  Construction 

□  Computer  Entry  Management 

□  Other 


□  Proper  Coordination 

□  Promptness  of  Actions 

□  Situation  Awareness  Maintenance 

□  Communication  Management 

□  Proper  Clearances  Issued 

□  Flight  Strip  Management 

□  Maintenance  of  Separation 


5.  What  was  most  difficult  for  you  to  accomplish  during  the  simulation  ? 

a)  (at  the)  ODAPS  Position  b)  (at  the)  "D"  Position 


NSC  SIMULATION:  CONTROLLER  FORM  II  (continued) 
CVSM  (Baseline)  CONDITIONS 


6.  If  you  could  change  something  about  the  last  simulation  run  (anything  at  all  about  the  traffic  scenario, 
aircraft,  procedures,  etc.) ,  what  would  it  be  ? 

7.  Did  you  change  your  usual  control  and  work  strategies  in  any  way  in  order  to  work  the  traffic  in  the 
last  simulation?  Ifso,  how?  What  did  you  do  differently? 

8.  Based  upon  your  experience  with  the  traffic  load  during  the  last  simulation  run,  what  procedures 
would  have  to  be  changed  and/or  implemented  in  order  for  you  to  continue  to  be  comfortable  about 
working  this  same  traffic  but  under  reduced  vertical  separation  minima  (RVSM)? 

9.  What  was  your  primary  safety  concern  considering  traffic,  events,  and  procedures  in  the  last 
simulation  run? 
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NSC  SIMULATION:  CONTROLLER  FORM  II  (continued) 
RVSM  CONDITIONS 


6.  If  you  could  change  something  about  the  last  simulation  run  (anything  at  all  about  the  traffic  scenario, 
aircraft,  procedures,  etc.) ,  what  would  it  be  ? 

7.  Did  you  change  your  usual  control  and  work  strategies  in  any  way  in  order  to  work  the  traffic  with 
RVSM?  If  so,  how?  What  did  you  do  differently? 

8.  Based  upon  your  experience  with  RVSM  in  the  last  simulation  run,  what  procedures  (equipment) 
would  have  to  be  changed  and/or  implemented  in  order  for  you  to  continue  to  be  comfortable  about 
transitioning  this  traffic  ? 

9.  What  was  your  primary  safety  concern  considering  traffic,  events,  and  procedures  in  the  last 
simulation  run? 
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Supplementary  Background  Information 

(to  be  completed  by  all  simulation  participants) 


Controller  #  _ 

(Last  four  digits  of  your  Social  Security  Number)  _ 

1 .  Please  indicate  the  total  time  you  have  worked  as  a  full  performance  level  controller  (in  any 

control  position,  not  only  oceanic): 

YEARS _  MONTHS _ 

2.  Please  indicate  the  total  time  you  have  worked  as  a  full  performance  level  oceanic  radar 

controller  (only  oceanic  experience): 

YEARS _  MONTHS _ 

3.  Please  indicate  the  total  time  you  have  worked  as  a  full  performance  level  oceanic  radar 

controller  with  North  Atlantic  traffic  (as  opposed  to  Pacific  traffic): 

YEARS _  MONTHS 

4.  Please  indicate  how  long  it  has  been  since  you  last  controlled  oceanic  radar  traffic  (i.e.,  indicate 

0  if  currently  active): 

YEARS _  MONTHS 

5.  Please  indicate  the  total  experience  you  have  as  a  trainer  of  controllers  (for  any  control  position): 

years  _____  MONTHS _ 

6.  Starting  with  your  current  facility,  please  list  all  FAA  facilities  which  you  have  worked  in 

throughout  your  career  as  a  controller: 

(1)_ _ (2) _ (3) _ (4) _ ^ _ 
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NSC  SIMULATION:  OBSERVER  FORM  II. 

INSTRUCTIONS: 

1 .  This  form  is  to  be  completed  by  all  observers  after  each  completed  simulation  run  in  the  NSC 
Phase  One  Simulation  activities.  The  form  consists  of  requests  for  information  regarding 
overall  experiences  and  judgments  about  the  just  completed  simulation  run. 

2.  The  RATING  NUMBERS  to  be  used  for  item  #4  are: 

1  =  Remarkably  good 

2  =  Moderately  good 

3  =  So-so 

4  =  Not  very  good 

5  =  Unusually  poor 

3.  Page  6  of  the  Form  consists  of  two  different  versions:  Version  1  is  to  be  filled  out  after 
CVSM  baseline  simulation  runs,  Version  2  is  to  be  filled  out  after  simulations  with  RVSM. 
The  difference  between  these  two  versions  is  that  version  2  elicits  additional,  RVSM-specific 
information. 
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NSC  SIMULATION:  OBSERVER  FORM  II. 
POST  SIMULATION  RUN  QUESTIONNAIRE 


Simulation  Condition:  □  CVSM  □  RVSM  □  RVSM-MTT  □  RVSM-CAT 
Observer  ID# _ Date: _ Simulation  Run  No.: _ 


la.  For  each  controller  please  estimate  the  overall  "D"  position  WORKLOAD  during  the  last  simulation 
(circle  one): 


Controller  SS#  tfour  digits) 

1  2  3 

VERY  LOW 

4 

5  6 

MODERATE 

7 

8 

9 

10 

VERY  HIGH 

Controller  SS#  tfour  digits) 

1  2  3 

VERY  LOW 

4 

5  6 

MODERATE 

7 

8 

9 

10 

VERY  HIGH 

lb  For  each  controller  please  estimate  the  overall  ODAPS  position  WORKLOAD 
simulation  (circle  one): 

during  the  last 

Controller  SS#  (four  digits)_ 

1  2  3 

VERY  LOW 

4 

5  6 

MODERATE 

7 

8 

9 

10 

VERY  HIGH 

Controller  SS#  tfour  digits) 

12  3 

VERY  LOW 

4 

5  6 

MODERATE 

7 

8 

9 

10 

VERY  HIGH 
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NSC  SIMULATION:  OBSERVER  FORM  II.  (continued) 


7  In  ynnr  opinion,  in  terms  of  REALISM,  how  real  was  the  PHYSICAL  SIMULATION 
ENVIRONMENT 

for  the  controllers  (eircle  one)? 

12  3  4 

5  6 

7  8  9  10 

VERY  LOW 

MODERATE 

VERY  HIGH 

7  Tn  yoiir  opinion,  in  terms  of  FUNCTIONAL  REALISM,  how  real  was  the  SIMULATED  TRAFFIC 

for  the 

controllers  (circle  one)? 

12  3  4 

5  6 

7  8  9  10 

VERY  LOW 

MODERATE 

VERY  HIGH 

4.  Please  judge  OVERALL  CONTROLLER  PERFORMANCE  during  the  most  recent  simulation  run. 

Using  the  RATING  NUMBERS  from  the  instruction  sheet,  insert  the  appropriate  rating  number  for  each 

of  the  positions  observed  into  the  box  next  to  each  of  the  following  factors: 

Controller  SS#  (4  digits) 

— 

ODAPS  POSITION 

D  POSITION 

□  Proper  Coordination 

□ 

Proper  Coordination 

□  Promptness  of  Actions 

□ 

Promptness  of  Actions 

□  Situation  Awareness  Maintenance 

□ 

Situation  Awareness  Maintenance 

□  Communication  Management 

□ 

Commimication  Management 

□  Proper  Message  Construction 

□ 

Proper  Clearances  Issued 

□  Computer  Entry  Management 

□ 

Flight  Strip  Management 

□  Other  □ 

Maintenance  of  Separation 
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Controller  SS#  (4  digits) 


ODAPS  POSITION 

D  POSITION 

□ 

Proper  Coordination 

□  Proper  Coordination 

□ 

Promptness  of  Actions 

□  Promptness  of  Actions 

□ 

Situation  Awareness  Maintenance 

□  Situation  Awareness  Maintenance 

□ 

Communication  Management 

□  Communication  Management 

□ 

Proper  Message  Construction 

□  Proper  Clearances  Issued 

□ 

Computer  Entry  Management 

□  Flight  Strip  Management 

□ 

Other  n  Maintenance  of  Separation 

5.  What  was  most  difficult  for  controllers  to  accomplish  during  this  simulation  ? 
a)  (at  the)  ODAPS  Position  b)  (at  the)  D  Position 


6.  If  you  could  change  something  about  the  last  simulation  run  (anything  at  all  about  the  traffic  scenario, 
aircraft,  procedures  etc.),  what  would  it  be  ? 


NSC  SIMULATION:  OBSERVER  FORM  II  (continued) 
CVSM  (Baseline)  CONDITIONS 


7.  Did  the  controllers  change  their  usual  control  and  work  strategies  in  any  way  in  order  to  work  the 
traffic  in  the  last  simulation?  If  so,  how?  What  did  they  do  differently? 

8.  Based  upon  your  observations  with  the  traffic  load  during  the  last  simulation  run,  what  procedures 
(equipment)  would  have  to  be  changed  and/or  implemented  in  order  for  the  controllers  to  be  comfortable 
about  working  this  same  traffic  but  under  reduced  vertical  separation  minima  (RVSM)? 

9.  What  was  your  primary  safety  concern  considering  traffic,  events,  and  procedures  in  the  last 
simulation  run? 
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NSC  SIMULATION:  OBSERVER  FORM  II  (continued) 

RVSM  CONDITIONS 

7.  Did  the  controllers  change  their  usual  control  and  work  strategies  in  any  way  in  order  to  work  the 
traffic  with  RVSM?  If  so,  how?  What  did  they  do  differently? 


I 


8.  Based  upon  your  observations  with  RVSM  in  the  last  simulation  run,  what  procedures  would  have  be 
changed  and/or  implemented  in  order  for  the  controllers  to  continue  to  be  comfortable  about  working 
this  traffic? 


9.  What  was  your  primary  safety  concern  considering  traffic,  events,  and  procedures  in  the  last 
simulation  run? 
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NSC  SIMULATION:  TECHNICAL  OBSERVER  "QUICK"  FORM. 

INSTRUCTIONS: 

1 .  This  form  is  to  be  completed  by  trained  technical  observers  during  each  simulation  run. 

2.  Observations  and  responses  will  be  recorded  in  1 5  minute  intervals.  A  new  (identical)  form 
should  be  used  for  each  1 5  minute  time  period. 

3.  Observations  will  be  made  on  this  form  primarily  for  the  "Radar-Position"  controller. 
However,  both  controllers  (Radar  and  Hand-off)  will  be  prompted  for  their  workload  rating  at 
15  minute  intervals  and  the  observer  will  make  "quick  evaluations"  about  performance  on 
both  positions. 

4.  The  rating  numbers  for  WORKLOAD  are  on  a  scale  from  1  to  10  with 

1  =  very  low  workload 
5  =  moderate  workload 
1 0  =  very  high  workload 

5.  The  rating  numbers  for  the  rest  of  the  QUICK  EVALUATIONS  are 

1  =  Remarkably  good 

2  =  Moderately  good 

3  =  So-so 

4  =  Not  very  good 

5  =  Unusually  poor 
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TECHNICAL  OBSERVER  "QUICK"  FORM 


INTERVAL:  TIME:  Simulation  Run  No. 

Date:  Sector  Number:  Observer  ID  CSS#) 

Prompt  for  workload  rating  from  the  controllers  every  15  minutes! 

WORKLOAD  RATING  -  "D"  Controller  :  12345678910 

WORKLOAD  RATING  -  "ODAPS"  Controller  :  12345678910 

RESTRICTIONS  ISSUED  (hash  marks): 

SPEED 

TIME 

ALTITUDE 

ROUTING 

ARINC  COMMUNICATIONS  GROUPINGS  (hash  marks): 

ONE  CLEARANCE 

TWO  AT  A  TIME 

THREE  AT  A  TIME 

FOUR  OR  MORE  AT  A  TIME 

OPERATIONAL  ERROR(s)  OBSERVED: 

□  Other 


Briefly  describe  what  happened: 


QUICK  ORSRRVER  EVALUATIONS  (circle  one) 

ODAPS  CONTROLLER  D  CONTROLLER 

WORKLOAD  RATING  ~ 

123456789  10 _ 1  2  3  4  5  6  7  8  9  10 

COORDINATION  &  COMMUNICATION 

1  2  3  4  5 _ _ _ 1  2  3  4  5 

TRAFFIC  MANAGEMENT  &  CONTROL  JUDGMENT 
12345  1234  5 

FLIGHT  STRIP  MANAGEMENT 

i  2  3  4  5  i  2  3  4  5 
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APPENDIX  D. 

SUMMARY  OF  POST-SIMULATION  RESPONSES 


TABLE  D-1.  POST-SIMULATION  RESPONSES 
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TABLE  D-2.  POST-SIMULATION  RESPONSES 
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TABLE  D-4.  POST-SIMULATION  RESPONSES  (continued) 
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